Entrega de numero 5 do Edital ATU-COLAB publicado em 2011

git-svn-id: http://repositorio.interlegis.gov.br/colab/trunk@6017 bee1b3ed-c3eb-0310-9994-b88e04532788

Entrega de numero 5 do Edital ATU-COLAB publicado em 2011
git-svn-id: http://repositorio.interlegis.gov.br/colab/trunk@6017 bee1b3ed-c3eb-0310-9994-b88e04532788
seocam
1 parent 89b44981
Showing 85 changed files with 6219 additions and 2342 deletions Show diff stats
MANIFEST.in
README.rst
TODO
TODO.rst
colab/api/handlers.py
colab/api/urls.py
colab/django.wsgi
colab/settings.py
colab/settings_local-dev.py
colab/signup.py
colab/socks.py
colab/solrutils.py
colab/static/css/screen.css
colab/static/img/COPYRIGHT
colab/static/img/changeset.png
colab/static/img/favicon.ico
colab/static/img/logo_small.png
colab/static/img/plus.png
colab/static/img/thread.png
colab/static/img/thumbs_up.jpg
@@ -0,0 +1,5 @@
+include colab/templates/*.html
+include colab/super_archives/templates/*.html
+include colab/super_archives/fixtures/initial_data.json
+recursive-include etc *
+recursive-include colab/static *
 \ No newline at end of file
@@ -0,0 +1,106 @@
+Installation instructions for Ubuntu 10.04
+-------------------------------------------
+
+* Install Apache2 with WSGI support:
+
+  * apt-get install apache2 libapache2-mod-wsgi
+
+* Install dependencies to compile psycopg2:
+
+  * apt-get build-dep python-psycopg2
+
+* Install Python PIP and update it:
+  
+  * apt-get install python-pip
+  * pip install -U pip
+
+* Install python virtualenv:
+  
+  * pip install virtualenv 
+
+* Create a virtualenv for the deploy
+ 
+  * mkdir /usr/local/django/
+  * virtualenv /usr/local/django/colab/
+
+* Download the colab src code:
+
+  * hg clone https://bitbucket.org/seocam/atu-colab /usr/local/src/colab/
+
+* Install the django site:
+
+  * pip install /usr/local/src/colab -E /usr/local/django/colab/
+
+* Configure your database settings in /usr/local/django/colab/lib/python2.6/site-packages/settings_local.py
+  
+* Enable the colab site on apache and reload it:
+
+  * ln -s /usr/local/django/colab/apache-site/colab /etc/apache2/sites-available
+  * a2ensite colab
+  * service apache2 restart
+  
+
+Configuring server to send emails
+----------------------------------
+
+* Install postfix and mailutils:
+ 
+  * apt-get install mailutils postfix
+
+* Update the file /etc/aliases adding users that should receive root's messages and run the update command:
+
+  * newaliases
+
+
+Cron job to import emails
+---------------------------
+
+* Install sshfs:
+  
+  * apt-get install sshfs autofs
+  
+* Create SSH keys. You should use a password but this tutorial won't cover it (if you use you will need to install and configure keychain process to be able to proceed):
+
+  * ssh-keygen
+  
+* Copy the content of your key (/root/.ssh/id_rsa.pub) to the file /root/.ssh/authorized_keys on the mailinglist server.
+
+* Append the following content to /etc/auto.master file:
+
+  * /usr/local/django/colab/mnt /usr/local/django/colab/autofs/listas --timeout=600,--ghost
+
+* Restart autofs:
+
+  * service autofs restart
+  
+* Link cron script into /etc/cron.d/ folder:
+
+  * ln -s /usr/local/django/colab/cron.d/colab_import_emails /etc/cron.d/ 
+  
+* From now on the emails should be imported every minute
+
+
+Cron job to reindex Solr
+-------------------------
+
+* Install wget:
+  
+  * apt-get install wget
+  
+* Link cron script into /etc/cron.d/ folder:
+  
+  * ln -s /usr/local/django/colab/cron.d/colab_solr_reindex /etc/cron.d/
+  
+* From now on delta reindex should run every 10 minutes and full reindex once a day. 
+
+
+Updating an installed version
+------------------------------
+
+* Update the source code:
+  
+  * cd /usr/local/src/colab/
+  * hg pull
+  * hg up
+  * pip install /usr/local/src/colab/ -E /usr/local/django/colab/ -U
+  * service apache2 restart
 \ No newline at end of file
@@ -0,0 +1,64 @@
+TODO
+-----
+
+* Sergio: Permitir que usuario atualize nome e sobrenome
+
+* Jean: Pagina Reporte um problema
+* Jean: Pagina Contribua
+
+* Yure: Adicionar data do ultimo import de emails no footer
+* Yure: Detectar links no conteudo e exibi-los como tal
+* Yure: BUG: Display of HTML emails are wrong
+* Yure: Cadastrar usuário em lista pelo formulario de cadastro
+* Arquivo "search.html" existente em "atu-colab/colab/templates" pode ser melhorado com relação ao conteúdo repetitivo dos "Tipos" exibidos no "Filtro" da página
+* Criar validador de urls para twitter, facebook e página pessoal do user profile
+
+* Configurar ADMINS no arquivo settings_local.py
+* HTTPS para o trac, subversion e colab
+* Autorizar usuarios a commitar no svn pelo django.contrib.admin
+* Enviar emails para usuarios pedindo que se cadastrem no novo colab
+* Timezones no trac/colab/solr nao estao compativeis
+
+* Template de login nao exibe corretamente no firefox/linux
+* Quando usuario se cadastra com email errado o email nunca eh validado,
+e o username fica preso 'pra sempre'. 
+* Nome dos usuarios errado nos emails que vem do Solr
+* Adicionar ordering na busca
+* Criar tipo usuario no solr 
+* Substituir sistema de cadastro por django-registration
+* Utilizar pysolr para efetuar queries no Solr
+* Melhorar buscas (case insensitive match, palavras com acentos)
+* Indexar nome do repositorio como campo e exibi-lo no titulo dos changesets retornados
+* Criacao de repositorios distribuidos pela interface do colab
+* Link para última msg recebida na thread
+* Fazer thread querysets ter um objeto (most_relevant_message)
+* Implementar enviar email
+* BUG: alguns subjects comecam e terminam com [] fazendo com que a RE de limpeza apague todo o subject.
+* BUG: mensagens importadas por listas erradas
+* Implementar login/signup usando LDAP
+* Claime email address
+* Merge emails dos usuarios
+* Implementar badge system
+* Melhorar filtros
+* Link do thread preview deve enviar para mensagem da thread (anchor) (Útil? discutir com Jean)
+* Tornar todas as strings traduziveis
+* Sugestão de como a divisão do edital seria melhor
+* Indexar anexos da wiki (using Tika http://wiki.apache.org/solr/ExtractingRequestHandler)
+* Filtrar usando calendario (como google analytics)
+* Melhorar relevancia das buscas usando dismax queryparser
+* Chat estilo Gmail
+* Sistema de gerencia de conteúdo
+* Versao mobile
+* Exibir discussões relacionadas na barra da direita das discussões
+* Migração e reorganização do conteúdo do trac/wiki para o novo colab
+* Ao importar mensagem sem subject enviar email avisando o usuario que ele esta enviando um email sem o campos "Assunto"
+* Remover ou ocultar trechos da mensagem que iniciem com ">" assim como o Gmail.
+* Contar page views no trac (ticket, wiki e changeset) e utiliza-los para rankear paginas nas buscas
+* Mostrar highlight nas buscas
+* Sistema de tags para as mensagens
+* Pagina home para cada lista com os mesmo filtros da home atual
+* Permitir que usuario entre e saia de listas ao editar perfil
+* Planet Interlegis (agregador de blogs)
+* Filtros especificos para tipos diferentes na busca
+* Indice criado manualmente. Automatizar:
+  * create index super_archives_message_body_idx ON super_archives_message ((substring(body,0,1024)));
  
+from django.core.cache import cache
 from django.db import IntegrityError
 from django.core.exceptions import ObjectDoesNotExist
 from django.contrib.auth.decorators import login_required
@@ -6,7 +7,7 @@ from django.contrib.auth.decorators import login_required
 from piston.utils import rc
 from piston.handler import BaseHandler
  
-from super_archives.models import Message
+from colab.super_archives.models import Message, PageHit
  
  
 class VoteHandler(BaseHandler):
@@ -38,5 +39,33 @@ class VoteHandler(BaseHandler):
         return rc.DELETED
  
  
+class CountHandler(BaseHandler):
+    allowed_methods = ('POST')
+
+    def create(self, request):
+        """Add one page view for the given url"""
+        
+        # If missing the path_info argument we can't do anything
+        path_info = request.POST.get('path_info')
+        if not path_info:
+            return rc.BAD_REQUEST
+
+        # Here we cache the user's IP to ensure that the same
+        #   IP won't hit the same page again for while
+        ip_addr = request.META.get('REMOTE_ADDR')
+        page_hits_cache = cache.get('page_hits', {})
+        duplicate = page_hits_cache.get(path_info, {}).get(ip_addr)
  
+        if duplicate:
+            return rc.DUPLICATE_ENTRY
+        else:
+            page_hits_cache.update({path_info: {ip_addr: True }})
+            cache.set('page_hits', page_hits_cache)
+        
+        # Everything ok, so just increment the page count
+        page_hit = PageHit.objects.get_or_create(url_path=path_info)[0]
+        page_hit.hit_count += 1
+        page_hit.save()
+        
+        return rc.CREATED
  
@@ -2,11 +2,13 @@ from django.conf.urls.defaults import patterns, include, url
  
 from piston.resource import Resource
  
-from api.handlers import VoteHandler
+from colab.api.handlers import VoteHandler, CountHandler
  
  
 vote_handler = Resource(VoteHandler)
+count_handler = Resource(CountHandler)
  
 urlpatterns = patterns('',
     url(r'message/(?P<message_id>\d+)/vote$', vote_handler),
+    url(r'hit/$', count_handler),
 )
 \ No newline at end of file
@@ -1,27 +0,0 @@
-import sys
-import site
-import os
-
-app_path = os.path.abspath(os.path.dirname(__file__))
-app_parent_path = os.path.abspath(os.path.join(app_path, os.path.pardir))
-
-vepath = '/home/seocam/.virtualenvs/colab/lib/python2.6/site-packages'
-
-prev_sys_path = list(sys.path)
-# add the site-packages of our virtualenv as a site dir
-site.addsitedir(vepath)
-# add the app's directory to the PYTHONPATH
-sys.path.append(app_path)
-sys.path.append(app_parent_path)
-
-# reorder sys.path so new directories from the addsitedir show up first
-new_sys_path = [p for p in sys.path if p not in prev_sys_path]
-for item in new_sys_path:
-    sys.path.remove(item)
-sys.path[:0] = new_sys_path
-
-# import from down here to pull in possible virtualenv django install
-from django.core.handlers.wsgi import WSGIHandler
-os.environ['DJANGO_SETTINGS_MODULE'] = 'colab.settings'
-application = WSGIHandler()
-
@@ -5,8 +5,12 @@ import os.path
 DEBUG = True
 TEMPLATE_DEBUG = DEBUG
  
+if DEBUG:
+    import logging
+    logging.root.setLevel(logging.DEBUG)
+
+
 ADMINS = (
-    # ('Your Name', 'your_email@example.com'),
 )
  
 PROJECT_PATH = os.path.abspath(os.path.dirname(__file__))
@@ -22,7 +26,7 @@ LOGIN_URL = &#39;/login/&#39;
 # timezone as the operating system.
 # If running in a Windows environment this must be set to the same as your
 # system time zone.
-TIME_ZONE = 'America/Chicago'
+TIME_ZONE = 'America/Sao_Paulo'
  
 # Language code for this installation. All choices can be found here:
 # http://www.i18nguy.com/unicode/language-identifiers.html
@@ -39,12 +43,12 @@ USE_I18N = True
 USE_L10N = True
  
 MEDIA_ROOT = os.path.join(PROJECT_PATH, os.pardir, 'site_media/media')
-MEDIA_URL = '/site_media/media/'
+MEDIA_URL = '/media/'
  
 STATIC_ROOT = os.path.join(PROJECT_PATH, os.pardir, 'site_media/static')
-STATIC_URL = '/site_media/static/'
+STATIC_URL = '/static/'
  
-#ADMIN_MEDIA_PREFIX = '/site_media/static/admin/'
+ADMIN_MEDIA_PREFIX = '/static/admin/'
  
 # Additional locations of static files
 STATICFILES_DIRS = (
@@ -66,12 +70,23 @@ TEMPLATE_LOADERS = (
 #     'django.template.loaders.eggs.Loader',
 )
  
+TEMPLATE_CONTEXT_PROCESSORS = (
+    'django.contrib.auth.context_processors.auth',
+    'django.core.context_processors.debug',
+    'django.core.context_processors.i18n',
+    'django.core.context_processors.media',
+    'django.core.context_processors.static',
+    'django.contrib.messages.context_processors.messages',
+    'django.core.context_processors.request',
+)
+
 MIDDLEWARE_CLASSES = (
     'django.middleware.common.CommonMiddleware',
     'django.contrib.sessions.middleware.SessionMiddleware',
     'django.middleware.csrf.CsrfViewMiddleware',
     'django.contrib.auth.middleware.AuthenticationMiddleware',
     'django.contrib.messages.middleware.MessageMiddleware',
+    #'debug_toolbar.middleware.DebugToolbarMiddleware',
 )
  
 ROOT_URLCONF = 'colab.urls'
@@ -91,17 +106,18 @@ INSTALLED_APPS = (
     'django.contrib.messages',
     'django.contrib.staticfiles',
     # Uncomment the next line to enable the admin:
-    # 'django.contrib.admin',
+    'django.contrib.admin',
     # Uncomment the next line to enable admin documentation:
     # 'django.contrib.admindocs',
  
     # Not standard apps
     'south',
-    'registration',
+    'cliauth',
+    #'debug_toolbar',
  
     # My apps
-    'super_archives',
-    'api',
+    'colab.super_archives',
+    'colab.api',
 )
  
 # A sample logging configuration. The only tangible logging
@@ -115,7 +131,8 @@ LOGGING = {
     'handlers': {
         'mail_admins': {
             'level': 'ERROR',
-            'class': 'django.utils.log.AdminEmailHandler'
+            'class': 'django.utils.log.AdminEmailHandler',
+            'include_html': True,
         }
     },
     'loggers': {
@@ -127,5 +144,18 @@ LOGGING = {
     }
 }
  
+SERVER_EMAIL = '"Colab Interlegis" <noreply@interlegis.leg.br>'
+EMAIL_HOST_USER = SERVER_EMAIL
+
+#SOLR_HOSTNAME = 'solr.interlegis.leg.br'
+SOLR_HOSTNAME = '10.1.2.154'
+SOLR_PORT = '8080'
+SOLR_SELECT_PATH = '/solr/select'
+
+SOLR_COLAB_URI = 'http://colab.interlegis.gov.br'
+SOLR_BASE_QUERY = """
+    (Type:changeset OR Type:ticket OR Type:wiki OR Type:thread)
+"""
+
 from settings_local import *
  
@@ -0,0 +1,18 @@
+
+DEBUG = True
+TEMPLATE_DEBUG = DEBUG
+
+ADMINS = (
+)
+
+MANAGERS = ADMINS
+
+DATABASES = {
+    'default': {
+        'ENGINE': 'django.db.backends.sqlite3', 
+	'NAME': 'colab.db',
+    }
+}
+
+# Make this unique, and don't share it with anybody.
+SECRET_KEY = ')(jksdfhsjkadfhjkh234ns!8fqu-1186h$vuj'
@@ -0,0 +1,50 @@
+#!/usr/bin/env python
+# encoding: utf-8
+
+from django.conf import settings
+from django.utils.html import strip_tags
+from django.utils.translation import ugettext as _
+from django.core.mail import EmailMultiAlternatives
+from django.template.loader import render_to_string
+
+
+def send_verification_email(request, user):
+    
+    subject = _(u'Colab: Verificação de email')
+    from_ = settings.SERVER_EMAIL
+    to = user.email
+    
+    email_data = {
+        'hash': user.profile.verification_hash,
+        'server_name': request.get_host(),
+    }
+
+    html_content = render_to_string('email_signup-email-confirmation.html', 
+                                     email_data)
+    text_content = strip_tags(html_content)
+    email_msg = EmailMultiAlternatives(subject, text_content, from_, [to])
+    email_msg.attach_alternative(html_content, 'text/html')
+    email_msg.send()
+
+
+def send_reset_password_email(request, user):
+
+    subject = _(u'Altereção de senha do Colab Interlegis')
+    from_ = settings.SERVER_EMAIL
+    to = user.email
+    
+    email_data = {
+        'hash': user.profile.verification_hash,
+        'server_name': request.get_host(),
+        'username': user.username,
+    }
+    
+    html_content = render_to_string('email_account-reset-password.html', 
+                                    email_data)
+    text_content = strip_tags(html_content)
+    
+    email_msg = EmailMultiAlternatives(subject, text_content, from_, [to])
+    email_msg.attach_alternative(html_content, 'text/html')
+    email_msg.send()
+    
+    
 \ No newline at end of file
@@ -0,0 +1,387 @@
+"""SocksiPy - Python SOCKS module.
+Version 1.00
+
+Copyright 2006 Dan-Haim. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+3. Neither the name of Dan Haim nor the names of his contributors may be used
+   to endorse or promote products derived from this software without specific
+   prior written permission.
+   
+THIS SOFTWARE IS PROVIDED BY DAN HAIM "AS IS" AND ANY EXPRESS OR IMPLIED
+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
+EVENT SHALL DAN HAIM OR HIS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA
+OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMANGE.
+
+
+This module provides a standard socket-like interface for Python
+for tunneling connections through SOCKS proxies.
+
+"""
+
+import socket
+import struct
+
+PROXY_TYPE_SOCKS4 = 1
+PROXY_TYPE_SOCKS5 = 2
+PROXY_TYPE_HTTP = 3
+
+_defaultproxy = None
+_orgsocket = socket.socket
+
+class ProxyError(Exception):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+class GeneralProxyError(ProxyError):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+class Socks5AuthError(ProxyError):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+class Socks5Error(ProxyError):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+class Socks4Error(ProxyError):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+class HTTPError(ProxyError):
+	def __init__(self, value):
+		self.value = value
+	def __str__(self):
+		return repr(self.value)
+
+_generalerrors = ("success",
+		   "invalid data",
+		   "not connected",
+		   "not available",
+		   "bad proxy type",
+		   "bad input")
+
+_socks5errors = ("succeeded",
+		  "general SOCKS server failure",
+		  "connection not allowed by ruleset",
+		  "Network unreachable",
+		  "Host unreachable",
+		  "Connection refused",
+		  "TTL expired",
+		  "Command not supported",
+		  "Address type not supported",
+		  "Unknown error")
+
+_socks5autherrors = ("succeeded",
+		      "authentication is required",
+		      "all offered authentication methods were rejected",
+		      "unknown username or invalid password",
+		      "unknown error")
+
+_socks4errors = ("request granted",
+		  "request rejected or failed",
+		  "request rejected because SOCKS server cannot connect to identd on the client",
+		  "request rejected because the client program and identd report different user-ids",
+		  "unknown error")
+
+def setdefaultproxy(proxytype=None,addr=None,port=None,rdns=True,username=None,password=None):
+	"""setdefaultproxy(proxytype, addr[, port[, rdns[, username[, password]]]])
+	Sets a default proxy which all further socksocket objects will use,
+	unless explicitly changed.
+	"""
+	global _defaultproxy
+	_defaultproxy = (proxytype,addr,port,rdns,username,password)
+	
+class socksocket(socket.socket):
+	"""socksocket([family[, type[, proto]]]) -> socket object
+	
+	Open a SOCKS enabled socket. The parameters are the same as
+	those of the standard socket init. In order for SOCKS to work,
+	you must specify family=AF_INET, type=SOCK_STREAM and proto=0.
+	"""
+	
+	def __init__(self, family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0, _sock=None):
+		_orgsocket.__init__(self,family,type,proto,_sock)
+		if _defaultproxy != None:
+			self.__proxy = _defaultproxy
+		else:
+			self.__proxy = (None, None, None, None, None, None)
+		self.__proxysockname = None
+		self.__proxypeername = None
+	
+	def __recvall(self, bytes):
+		"""__recvall(bytes) -> data
+		Receive EXACTLY the number of bytes requested from the socket.
+		Blocks until the required number of bytes have been received.
+		"""
+		data = ""
+		while len(data) < bytes:
+			data = data + self.recv(bytes-len(data))
+		return data
+	
+	def setproxy(self,proxytype=None,addr=None,port=None,rdns=True,username=None,password=None):
+		"""setproxy(proxytype, addr[, port[, rdns[, username[, password]]]])
+		Sets the proxy to be used.
+		proxytype -	The type of the proxy to be used. Three types
+				are supported: PROXY_TYPE_SOCKS4 (including socks4a),
+				PROXY_TYPE_SOCKS5 and PROXY_TYPE_HTTP
+		addr -		The address of the server (IP or DNS).
+		port -		The port of the server. Defaults to 1080 for SOCKS
+				servers and 8080 for HTTP proxy servers.
+		rdns -		Should DNS queries be preformed on the remote side
+				(rather than the local side). The default is True.
+				Note: This has no effect with SOCKS4 servers.
+		username -	Username to authenticate with to the server.
+				The default is no authentication.
+		password -	Password to authenticate with to the server.
+				Only relevant when username is also provided.
+		"""
+		self.__proxy = (proxytype,addr,port,rdns,username,password)
+	
+	def __negotiatesocks5(self,destaddr,destport):
+		"""__negotiatesocks5(self,destaddr,destport)
+		Negotiates a connection through a SOCKS5 server.
+		"""
+		# First we'll send the authentication packages we support.
+		if (self.__proxy[4]!=None) and (self.__proxy[5]!=None):
+			# The username/password details were supplied to the
+			# setproxy method so we support the USERNAME/PASSWORD
+			# authentication (in addition to the standard none).
+			self.sendall("\x05\x02\x00\x02")
+		else:
+			# No username/password were entered, therefore we
+			# only support connections with no authentication.
+			self.sendall("\x05\x01\x00")
+		# We'll receive the server's response to determine which
+		# method was selected
+		chosenauth = self.__recvall(2)
+		if chosenauth[0] != "\x05":
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		# Check the chosen authentication method
+		if chosenauth[1] == "\x00":
+			# No authentication is required
+			pass
+		elif chosenauth[1] == "\x02":
+			# Okay, we need to perform a basic username/password
+			# authentication.
+			self.sendall("\x01" + chr(len(self.__proxy[4])) + self.__proxy[4] + chr(len(self.proxy[5])) + self.__proxy[5])
+			authstat = self.__recvall(2)
+			if authstat[0] != "\x01":
+				# Bad response
+				self.close()
+				raise GeneralProxyError((1,_generalerrors[1]))
+			if authstat[1] != "\x00":
+				# Authentication failed
+				self.close()
+				raise Socks5AuthError,((3,_socks5autherrors[3]))
+			# Authentication succeeded
+		else:
+			# Reaching here is always bad
+			self.close()
+			if chosenauth[1] == "\xFF":
+				raise Socks5AuthError((2,_socks5autherrors[2]))
+			else:
+				raise GeneralProxyError((1,_generalerrors[1]))
+		# Now we can request the actual connection
+		req = "\x05\x01\x00"
+		# If the given destination address is an IP address, we'll
+		# use the IPv4 address request even if remote resolving was specified.
+		try:
+			ipaddr = socket.inet_aton(destaddr)
+			req = req + "\x01" + ipaddr
+		except socket.error:
+			# Well it's not an IP number,  so it's probably a DNS name.
+			if self.__proxy[3]==True:
+				# Resolve remotely
+				ipaddr = None
+				req = req + "\x03" + chr(len(destaddr)) + destaddr
+			else:
+				# Resolve locally
+				ipaddr = socket.inet_aton(socket.gethostbyname(destaddr))
+				req = req + "\x01" + ipaddr
+		req = req + struct.pack(">H",destport)
+		self.sendall(req)
+		# Get the response
+		resp = self.__recvall(4)
+		if resp[0] != "\x05":
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		elif resp[1] != "\x00":
+			# Connection failed
+			self.close()
+			if ord(resp[1])<=8:
+				raise Socks5Error(ord(resp[1]),_generalerrors[ord(resp[1])])
+			else:
+				raise Socks5Error(9,_generalerrors[9])
+		# Get the bound address/port
+		elif resp[3] == "\x01":
+			boundaddr = self.__recvall(4)
+		elif resp[3] == "\x03":
+			resp = resp + self.recv(1)
+			boundaddr = self.__recvall(resp[4])
+		else:
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		boundport = struct.unpack(">H",self.__recvall(2))[0]
+		self.__proxysockname = (boundaddr,boundport)
+		if ipaddr != None:
+			self.__proxypeername = (socket.inet_ntoa(ipaddr),destport)
+		else:
+			self.__proxypeername = (destaddr,destport)
+	
+	def getproxysockname(self):
+		"""getsockname() -> address info
+		Returns the bound IP address and port number at the proxy.
+		"""
+		return self.__proxysockname
+	
+	def getproxypeername(self):
+		"""getproxypeername() -> address info
+		Returns the IP and port number of the proxy.
+		"""
+		return _orgsocket.getpeername(self)
+	
+	def getpeername(self):
+		"""getpeername() -> address info
+		Returns the IP address and port number of the destination
+		machine (note: getproxypeername returns the proxy)
+		"""
+		return self.__proxypeername
+	
+	def __negotiatesocks4(self,destaddr,destport):
+		"""__negotiatesocks4(self,destaddr,destport)
+		Negotiates a connection through a SOCKS4 server.
+		"""
+		# Check if the destination address provided is an IP address
+		rmtrslv = False
+		try:
+			ipaddr = socket.inet_aton(destaddr)
+		except socket.error:
+			# It's a DNS name. Check where it should be resolved.
+			if self.__proxy[3]==True:
+				ipaddr = "\x00\x00\x00\x01"
+				rmtrslv = True
+			else:
+				ipaddr = socket.inet_aton(socket.gethostbyname(destaddr))
+		# Construct the request packet
+		req = "\x04\x01" + struct.pack(">H",destport) + ipaddr
+		# The username parameter is considered userid for SOCKS4
+		if self.__proxy[4] != None:
+			req = req + self.__proxy[4]
+		req = req + "\x00"
+		# DNS name if remote resolving is required
+		# NOTE: This is actually an extension to the SOCKS4 protocol
+		# called SOCKS4A and may not be supported in all cases.
+		if rmtrslv==True:
+			req = req + destaddr + "\x00"
+		self.sendall(req)
+		# Get the response from the server
+		resp = self.__recvall(8)
+		if resp[0] != "\x00":
+			# Bad data
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		if resp[1] != "\x5A":
+			# Server returned an error
+			self.close()
+			if ord(resp[1]) in (91,92,93):
+				self.close()
+				raise Socks4Error((ord(resp[1]),_socks4errors[ord(resp[1])-90]))
+			else:
+				raise Socks4Error((94,_socks4errors[4]))
+		# Get the bound address/port
+		self.__proxysockname = (socket.inet_ntoa(resp[4:]),struct.unpack(">H",resp[2:4])[0])
+		if rmtrslv != None:
+			self.__proxypeername = (socket.inet_ntoa(ipaddr),destport)
+		else:
+			self.__proxypeername = (destaddr,destport)
+	
+	def __negotiatehttp(self,destaddr,destport):
+		"""__negotiatehttp(self,destaddr,destport)
+		Negotiates a connection through an HTTP server.
+		"""
+		# If we need to resolve locally, we do this now
+		if self.__proxy[3] == False:
+			addr = socket.gethostbyname(destaddr)
+		else:
+			addr = destaddr
+		self.sendall("CONNECT " + addr + ":" + str(destport) + " HTTP/1.1\r\n" + "Host: " + destaddr + "\r\n\r\n")
+		# We read the response until we get the string "\r\n\r\n"
+		resp = self.recv(1)
+		while resp.find("\r\n\r\n")==-1:
+			resp = resp + self.recv(1)
+		# We just need the first line to check if the connection
+		# was successful
+		statusline = resp.splitlines()[0].split(" ",2)
+		if statusline[0] not in ("HTTP/1.0","HTTP/1.1"):
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		try:
+			statuscode = int(statusline[1])
+		except ValueError:
+			self.close()
+			raise GeneralProxyError((1,_generalerrors[1]))
+		if statuscode != 200:
+			self.close()
+			raise HTTPError((statuscode,statusline[2]))
+		self.__proxysockname = ("0.0.0.0",0)
+		self.__proxypeername = (addr,destport)
+	
+	def connect(self,destpair):
+		"""connect(self,despair)
+		Connects to the specified destination through a proxy.
+		destpar - A tuple of the IP/DNS address and the port number.
+		(identical to socket's connect).
+		To select the proxy server use setproxy().
+		"""
+		# Do a minimal input check first
+		if (type(destpair) in (list,tuple)==False) or (len(destpair)<2) or (type(destpair[0])!=str) or (type(destpair[1])!=int):
+			raise GeneralProxyError((5,_generalerrors[5]))
+		if self.__proxy[0] == PROXY_TYPE_SOCKS5:
+			if self.__proxy[2] != None:
+				portnum = self.__proxy[2]
+			else:
+				portnum = 1080
+			_orgsocket.connect(self,(self.__proxy[1],portnum))
+			self.__negotiatesocks5(destpair[0],destpair[1])
+		elif self.__proxy[0] == PROXY_TYPE_SOCKS4:
+			if self.__proxy[2] != None:
+				portnum = self.__proxy[2]
+			else:
+				portnum = 1080
+			_orgsocket.connect(self,(self.__proxy[1],portnum))
+			self.__negotiatesocks4(destpair[0],destpair[1])
+		elif self.__proxy[0] == PROXY_TYPE_HTTP:
+			if self.__proxy[2] != None:
+				portnum = self.__proxy[2]
+			else:
+				portnum = 8080
+			_orgsocket.connect(self,(self.__proxy[1],portnum))
+			self.__negotiatehttp(destpair[0],destpair[1])
+		elif self.__proxy[0] == None:
+			_orgsocket.connect(self,(destpair[0],destpair[1]))
+		else:
+			raise GeneralProxyError((4,_generalerrors[4]))
@@ -0,0 +1,264 @@
+#!/usr/bin/env python
+# encoding: utf-8
+
+import math
+import json
+import urllib
+import socket
+import logging
+import httplib
+
+from dateutil.parser import parse as parse_timestamp
+
+from django.conf import settings
+
+from colab.super_archives.models import EmailAddress
+
+
+def build_query(user_query, filters=None):
+    """Build the query that will be sent to Solr"""    
+
+    if not user_query:
+        user_query = '*'
+
+    query = settings.SOLR_BASE_QUERY.strip() + ' AND ' + user_query
+    if filters:
+        query = "(%s)" % query
+
+        for (key, value) in filters.items():
+            if value:
+                query += " AND %s:%s" % (key, value)
+    
+    logging.info(query)
+    return query.encode('utf-8')
+
+
+def parse_document_timestamps(doc, date_attrs=('modified', 'created')):
+    """Converts the `modified' and `created' dates from
+    ISO 8601 format to a date time object for the given 
+    document.
+    
+    """
+    
+    for date in date_attrs:
+        date_str = doc.get(date)
+        try:
+            date_obj = parse_timestamp(date_str)
+        except ValueError:
+            logging.error('Error trying to parse "%s"', date_str)
+            date_obj = None
+        doc.update({date: date_obj})
+    
+    return doc
+
+
+def get_document_url(doc):
+    """Set the url attribute for a document using the path_string.
+    In case the resource comes from an external domain it will
+    be prepended to this URL.
+
+    """
+    doc_type = doc.get('Type')
+    
+    url = ''
+    if doc_type in ('ticket', 'wiki', 'changeset'):
+        url += settings.SOLR_COLAB_URI
+    
+    url += doc.get('path_string', '') 
+    doc.update({'url': url})
+    
+    return doc
+    
+
+def get_document_from_addr(doc):
+    """Get a EmailAddress instance for the given document if 
+    its available.
+    
+    """
+    
+    username = doc.get('Creator')
+    from_addresses = EmailAddress.objects.filter(user__username=username)
+    if username and from_addresses:
+        doc.update({'from_address': from_addresses[0]})
+    
+
+def add_attrs_to_doc(doc):
+    """Wraps the call of functions that adds or modifies keys
+    of the giving doc (which should be a dict).
+    
+    """
+    get_document_url(doc)
+    parse_document_timestamps(doc)
+    get_document_from_addr(doc)
+
+
+class SolrPaginator(list):
+    
+    def __init__(self, response_dict, current_page):
+        super(SolrPaginator, self).__init__()
+        
+        responseHeader = response_dict.get('responseHeader', {}) 
+        response = response_dict.get('response', {})
+        request_params = responseHeader.get('params', {})
+        
+        docs = response.get('docs', [])
+        self.extend(docs)
+
+        self.QTime = int(responseHeader.get('QTime', 1)) / 1000.0
+       
+        self.per_page = int(request_params.get('rows', 10))
+        self.numFound = int(response.get('numFound', 0))
+        self.page_num = current_page
+    
+        self.num_of_pages = int(math.ceil(self.numFound / float(self.per_page)))
+        
+        self.has_previous = self.page_num > 1
+        if self.has_previous: 
+            self.previous_page_number = self.page_num - 1
+        else:
+            self.previous_page_number = None
+        
+        self.has_next = self.page_num < self.num_of_pages
+        if self.has_next:
+            self.next_page_number = self.page_num + 1        
+        else:
+            self.next_page_number = None
+    
+    @property
+    def last_page(self):
+        return self.num_of_pages
+
+
+def select(query, results_per_page=None, page_number=None, sort=None, fields=None, link_attrs=True):
+    """Perform a select in a Solr instance using the configuration
+    set in settings.py.
+    
+    """
+    
+    data = {
+        'q': query, 
+        'wt': 'json',
+    }
+    
+    # Number of results per page
+    if results_per_page:
+        data.update({'rows': results_per_page})
+        
+        # Page number
+        if page_number:
+            data.update({
+                'start': (page_number - 1) * results_per_page,
+            })
+            
+    # Sort order
+    if sort:
+        data.update({
+            'sort': sort,
+        })
+    
+    # Only select those fields
+    if fields:
+        data.update({
+            'fl': ','.join(fields),
+        })
+    # First version of this was implemented using urllib2 and was
+    #   a milion times easier but unfortunatelly urllib2.urlopen
+    #   does not support http headers. Without setting http headers
+    #   for charset the solr server tries to decode utf-8 params
+    #   as ASCII causing it to crash. HTTPConnection deals with
+    #   encodings automagically.
+    solr_conn = httplib.HTTPConnection(settings.SOLR_HOSTNAME,          
+                                       settings.SOLR_PORT)
+    query_params = urllib.urlencode(data)
+    solr_select_uri = settings.SOLR_SELECT_PATH + '?' + query_params
+   
+    # Socks proxy configuration. Only required for development
+    #   if the solr server is behind a firewall. 
+    socks_server = getattr(settings, "SOCKS_SERVER", None)
+    if socks_server: 
+        import socks
+        logging.debug('Socks enabled: %s:%s', settings.SOCKS_SERVER,
+                                              settings.SOLR_PORT)
+
+        socks.setdefaultproxy(settings.SOCKS_TYPE, 
+                              settings.SOCKS_SERVER,
+                              settings.SOCKS_PORT)
+        socket.socket = socks.socksocket
+
+    try:
+        solr_conn.request('GET', solr_select_uri)
+        solr_response = solr_conn.getresponse()
+    except socket.error as err: 
+        solr_response = None
+        logging.exception(err)
+
+    if solr_response and solr_response.status == 200:
+        #TODO: Log error connecting to solr
+        solr_json_resp = solr_response.read()
+        solr_dict_resp = json.loads(solr_json_resp)
+    else:
+        solr_dict_resp = {}
+    
+    docs = solr_dict_resp.get('response', {}).get("docs", [])
+
+    if link_attrs:
+        # Loop over all documents adding or linking its information
+        #   with the data from this app or database
+        map(add_attrs_to_doc, docs)
+    
+    return solr_dict_resp
+    
+
+def get_latest_collaborations(number=10, username=None):
+    """Get the n documents recently modified that this username
+    has helped in somehow.
+    
+    """
+    
+    if username:
+        filters = {'collaborator': username}
+    else:
+        filters = None
+    
+    query = build_query('*', filters)
+    solr_response = select(
+        query=query, 
+        results_per_page=number, 
+        sort='modified desc'
+    )
+    
+    return solr_response.get('response', {}).get('docs', [])
+
+
+def count_types(sample=100, filters=None):
+    """Count the type of the last modifications returning the
+    results in dict.
+    
+    Example: {
+        'wiki' 30,
+        'thread': 40,
+        'ticket', 10,
+        'changeset' 20,
+    }
+    
+    """
+    
+    query = build_query('*', filters)
+    solr_response = select(
+        query=query,
+        results_per_page=sample,
+        sort='modified desc',
+        link_attrs=False,
+    )
+
+    docs = solr_response.get('response', {}).get('docs', [])
+
+    type_count = {}
+    for doc in docs:
+        doc_type = doc.get('Type')
+        count = type_count.get(doc_type, 0) + 1
+        type_count.update({doc_type: count})
+
+    return type_count
+    
+    
@@ -25,6 +25,9 @@ h4 {
     margin-bottom: 10px;
 }
  
+form {
+    display: inline;
+}
  
 #main-content {
     margin-bottom: 20px;
@@ -48,16 +51,24 @@ h4 {
     text-decoration: none;
 }
  
+.preview-message img {
+    margin-right: 5px;
+}
+
 /* Header */
  
 #header-searchbox {
-    width: 150px;
+    width: 290px;
+}
+
+#header-menu span {
+    line-height: 33px;
 }
  
 /* Thread view */
  
 .plus {
-    border: 1px dashed #ddd;
+    border: 1px dotted #ddd;
     margin: 0;
 }
  
@@ -72,8 +83,13 @@ h4 {
  
 .plus img {
     vertical-align: middle;
-    height: 40px;
     cursor: pointer;
+    margin-top: 8px;
+    margin-right: 20px;
+}
+
+.plus img:hover {
+    opacity: 0.8;
 }
  
 /* User Profile  */
@@ -125,11 +141,32 @@ input[type=&quot;password&quot;]  {
     color: #335;
 }
  
+.subject img {
+    margin-right: 5px;
+}
+
 .avatar {
     border: 1px solid #ddd;
     padding: 3px;
 }
  
+div.avatar-placeholder {
+    border: 1px solid #cccccc;
+}
+
+.avatar-image {
+    float: left;
+    margin-right: 10px;
+    height: 40px;
+    padding: 8px 12px 0;
+    background-color: white;
+}
+
+label.avatar-placeholder {
+    display: block;
+    margin-left: 6px;
+}
+
 .tag {
     background-color: #2183b3;
     padding: 2px 4px;
@@ -173,4 +210,16 @@ img.center {
  
 .filters li:before {
     content: "\00BB\0020";
-}
 \ No newline at end of file
+}
+
+
+.filters .legend li span {
+    display: inline-block;
+    width: 25px;
+}
+
+.filters .legend li:before {
+    content: '';
+}
+
+
@@ -0,0 +1,10 @@
+The icons listed bellow were copied from the Iconic icons package. The icons in this set were originally designed for the Franklin Street WordPress theme and are available under CC Attribution-Share Alike 3.0 license - http://creativecommons.org/licenses/by-sa/3.0/us/
+
+* ticket.png
+* changeset.png
+* thread.png
+* wiki.png 
+* x.png
+* plus.png
+
+The full Iconic package can be found here: https://github.com/downloads/somerandomdude/Iconic/iconic.zip
@@ -5,6 +5,7 @@ function vote_callback(msg_id, step) {
             return parseInt(count) + step;
         });
         jQuery('#msg-' + msg_id + ' .minus').toggleClass('hide');
+        jQuery('#vote-notification').addClass('hide');
     }
 }
  
@@ -21,6 +22,18 @@ function get_vote_ajax_dict(msg_id, type_) {
         url: "/api/message/" + msg_id + "/vote",
         type: type_,
         success: vote_callback(msg_id, step),
+        error: function (jqXHR, textStatus, errorThrown) {
+            
+            error_msg = '<b>Seu voto não foi computado.</b>'
+            if (jqXHR.status === 401) {
+                error_msg += ' Você deve estar autenticado para votar.';
+            } else {
+                error_msg += ' Erro desconhecido ao tentando votar.';
+            }
+            
+            jQuery('#vote-notification').html(error_msg).removeClass('hide');
+            scroll(0, 0);
+        }
     }
 }
  
@@ -35,7 +48,6 @@ function unvote(msg_id) {
 jQuery(document).ready(function() {
     jQuery('.email_message').each(function() {
         var msg_id = this.getAttribute('id').split('-')[1];
-        console.debug(msg_id);
         jQuery('.plus img', this).bind('click', function() {
             vote(msg_id);
         });
@@ -44,4 +56,12 @@ jQuery(document).ready(function() {
             return false;
         });
     });
-});
 \ No newline at end of file
+});
+
+function pagehit(path_info) {
+    jQuery.ajax({
+        url: '/api/hit/',
+        type: 'POST',
+        data: {'path_info': path_info},
+    });
+}
  
 from django.contrib import admin
-from super_archives.models import MailingList, MailingListMembership, \
-                                  Message, MessageMetadata, Vote
-
-admin.site.register(MailingList)
-admin.site.register(MailingListMembership)
-admin.site.register(Message)
-admin.site.register(MessageMetadata)
-admin.site.register(Vote)
+from colab.super_archives.models import Message, Thread, UserProfile
+
+class MessageAdmin(admin.ModelAdmin):
+    list_filter = ('spam', 'mailinglist', 'received_time', )
+    search_fields = (
+        'id',
+        'subject',
+        'subject_clean',
+        'body', 
+        'from_address__real_name',
+        'from_address__address',
+        'from_address__user__first_name',
+        'from_address__user__last_name',
+        'from_address__user__username',
+    )
+    readonly_fields = ('thread', 'from_address', 'mailinglist')
+
+
+class ThreadAdmin(admin.ModelAdmin):
+    list_filter = ('spam', 'mailinglist', 'message__received_time',)
+    search_fields = (
+        'id',
+        'subject_token',
+        'message__subject',
+        'message__subject_clean',
+        'message__from_address__real_name',
+        'message__from_address__address',
+        'message__from_address__user__first_name',
+        'message__from_address__user__last_name',
+        'message__from_address__user__username',
+    )
+    
+    readonly_fields = (
+        'mailinglist',
+        'subject_token',
+        'latest_message',
+        'score',
+    )
+    
+    fields = (
+        'mailinglist',
+        'subject_token',
+        'latest_message',
+        'score',
+        'spam',
+    )
+    
+
+admin.site.register(UserProfile)
+admin.site.register(Thread, ThreadAdmin)
+admin.site.register(Message, MessageAdmin)
+
@@ -0,0 +1,10 @@
+[
+    {
+        "pk": 1, 
+        "model": "auth.group", 
+        "fields": {
+            "name": "developer", 
+            "permissions": []
+        }
+    }
+]
-# -*- coding: utf8 -*-
+# -*- coding: utf-8 -*-
  
 from django import forms
 from django.core.exceptions import ValidationError
 from django.contrib.auth.models import User
 from django.contrib.auth.forms import UserCreationForm as UserCreationForm_
  
-from super_archives.validators import UniqueValidator    
+from colab.super_archives.validators import UniqueValidator    
  
 # XXX: I know that this code does not look nice AT ALL.
-#   probably it should be implemented using formsets instead of the hacking
-#   below. Feel free to improve it! :)
+#   probably it should be implemented using formsets instead of 
+#   the hack below. Feel free to improve it! :)
  
 # User fields
 username_field = UserCreationForm_().fields.get('username')
@@ -21,10 +21,10 @@ email_field = forms.EmailField(validators=[UniqueValidator(User, &#39;email&#39;)])
 institution_field = forms.CharField(max_length=120, label=u'Instituição', 
                                     required=False)
 role_field = forms.CharField(max_length=60, label='Função', required=False)
-twitter_field = forms.CharField(label=u'Twitter', required=False)
-facebook_field = forms.CharField(label=u'Facebook', required=False)
+twitter_field = forms.URLField(label=u'Twitter', required=False)
+facebook_field = forms.URLField(label=u'Facebook', required=False)
 google_talk_field = forms.EmailField(label=u'Google Talk', required=False)
-webpage_field = forms.CharField(label=u'Página Pessoal/Blog', required=False)
+webpage_field = forms.URLField(label=u'Página Pessoal/Blog', required=False)
  
  
 class UserCreationForm(UserCreationForm_):
 #!/usr/bin/env python
-# -*- encoding: utf8 -*-
+# -*- encoding: utf-8 -*-
  
 """Import emails from a mailman storage to the django database."""
  
 import os
 import re
+import sys
 import mailbox
 from optparse import make_option
  
@@ -13,8 +14,10 @@ from django.template.defaultfilters import slugify
 from django.core.exceptions import ObjectDoesNotExist
 from django.core.management.base import BaseCommand, CommandError
  
-from super_archives.models import MailingList, Message, Thread, EmailAddress
-from super_archives.management.commands.message import Message as CustomMessage
+from colab.super_archives.models import MailingList, Message, \
+                                        Thread, EmailAddress
+from colab.super_archives.management.commands.message import Message as \
+                                                             CustomMessage
  
  
 class Command(BaseCommand, object):
@@ -35,7 +38,14 @@ class Command(BaseCommand, object):
             help='Path of email archives to be imported. (default: %s)' % 
                                                     default_archives_path,
             default=default_archives_path),
-            
+        
+        make_option('--exclude-list',
+            dest='exclude_lists',
+            help=("Mailing list that won't be imported. It can be used many"    
+                  "times for more than one list."),
+            action='append',
+            default=None),
+        
         make_option('--all',
             dest='all',
             help='Import all messages (default: False)',
@@ -81,11 +91,12 @@ class Command(BaseCommand, object):
             key += 1
             yield key-1, mbox[key-1]
  
-    def get_emails(self, mailinglist_dir, all):
+    def get_emails(self, mailinglist_dir, all, exclude_lists):
         """Generator function that get the emails from each mailing
         list dump dirctory. If `all` is set to True all the emails in the 
         mbox will be imported if not it will just resume from the last 
-        message previously imported.
+        message previously imported. The lists set in `exclude_lists`
+        won't be imported.
  
         Yield: A tuple in the form: (mailing list name, email message).
  
@@ -101,6 +112,10 @@ class Command(BaseCommand, object):
             mbox_path = os.path.join(mailinglist_dir, mbox, mbox)
             mailinglist_name = mbox.split('.')[0]
  
+            # Check if the mailinglist is set not to be imported
+            if exclude_lists and mailinglist_name in exclude_lists:
+                continue
+            
             # Find the index of the last imported message
             if all:
                 n_msgs = 0
@@ -154,10 +169,7 @@ class Command(BaseCommand, object):
         email_addr = self.EMAIL_ADDR_CACHE.get(from_)
         if email_addr is None:
             email_addr = EmailAddress.objects.get_or_create(
-                address=from_,
-                real_name=real_name[:64]
-            )[0]
-            
+                address=from_)[0]
             self.EMAIL_ADDR_CACHE[from_] = email_addr
  
         if not email_addr.real_name and real_name:
@@ -179,16 +191,18 @@ class Command(BaseCommand, object):
         email.save()
  
     @transaction.commit_manually    
-    def import_emails(self, archives_path, all):
+    def import_emails(self, archives_path, all, exclude_lists=None):
         """Get emails from the filesystem from the `archives_path` 
         and store them into the database. If `all` is set to True all 
-        the filesystem storage will be imported otherwise the importation
-        will resume from the last message previously imported.
+        the filesystem storage will be imported otherwise the 
+        importation will resume from the last message previously 
+        imported. The lists set in `exclude_lists` won't be imported.
  
         """
  
         count = 0
-        for mailinglist_name, msg, index in self.get_emails(archives_path, all):
+        email_generator = self.get_emails(archives_path, all, exclude_lists)
+        for mailinglist_name, msg, index in email_generator:
             try:
                 self.save_email(mailinglist_name, msg, index)
             except:
@@ -206,6 +220,20 @@ class Command(BaseCommand, object):
     def handle(self, *args, **options):
         """Main command method."""
  
+        lock_file = '/var/lock/colab/import_emails.lock'
+        
+        # Already running, so quit
+        if os.path.exists(lock_file):
+            self.log(("This script is already running. (If your are sure it's "
+                     "not please delete the lock file in %s')") % lock_file)
+            sys.exit(0)
+    
+        if not os.path.exists(os.path.dirname(lock_file)):
+            os.mkdir(os.path.dirname(lock_file), 0755)
+        
+        run_lock = file(lock_file, 'w')
+        run_lock.close()
+        
         archives_path = options.get('archives_path')
         self.log('Using archives_path `%s`' % self.default_archives_path)
  
@@ -213,5 +241,8 @@ class Command(BaseCommand, object):
             raise CommandError('archives_path (%s) does not exist' %
                                                                 archives_path)
  
-        self.import_emails(archives_path, options.get('all'))
+        self.import_emails(archives_path, 
+                           options.get('all'), options.get('exclude_lists'))
+        
+        os.remove(lock_file)
  
 \ No newline at end of file
@@ -0,0 +1,255 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding model 'PageHit'
+        db.create_table('super_archives_pagehit', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('url_path', self.gf('django.db.models.fields.CharField')(unique=True, max_length=2048, db_index=True)),
+            ('hit_count', self.gf('django.db.models.fields.IntegerField')(default=0)),
+        ))
+        db.send_create_signal('super_archives', ['PageHit'])
+
+        # Adding model 'EmailAddress'
+        db.create_table('super_archives_emailaddress', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('user', self.gf('django.db.models.fields.related.ForeignKey')(related_name='emails', null=True, to=orm['auth.User'])),
+            ('address', self.gf('django.db.models.fields.EmailField')(unique=True, max_length=75)),
+            ('real_name', self.gf('django.db.models.fields.CharField')(max_length=64, blank=True)),
+            ('md5', self.gf('django.db.models.fields.CharField')(max_length=32, null=True)),
+        ))
+        db.send_create_signal('super_archives', ['EmailAddress'])
+
+        # Adding model 'UserProfile'
+        db.create_table('super_archives_userprofile', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('user', self.gf('django.db.models.fields.related.OneToOneField')(to=orm['auth.User'], unique=True)),
+            ('institution', self.gf('django.db.models.fields.CharField')(max_length=128, null=True)),
+            ('role', self.gf('django.db.models.fields.CharField')(max_length=128, null=True)),
+            ('twitter', self.gf('django.db.models.fields.CharField')(max_length=128, null=True)),
+            ('facebook', self.gf('django.db.models.fields.CharField')(max_length=128, null=True)),
+            ('google_talk', self.gf('django.db.models.fields.EmailField')(max_length=75, null=True)),
+            ('webpage', self.gf('django.db.models.fields.CharField')(max_length=256)),
+        ))
+        db.send_create_signal('super_archives', ['UserProfile'])
+
+        # Adding model 'MailingList'
+        db.create_table('super_archives_mailinglist', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('name', self.gf('django.db.models.fields.CharField')(max_length=80)),
+            ('email', self.gf('django.db.models.fields.EmailField')(max_length=75)),
+            ('description', self.gf('django.db.models.fields.TextField')()),
+            ('logo', self.gf('django.db.models.fields.files.FileField')(max_length=100)),
+            ('last_imported_index', self.gf('django.db.models.fields.IntegerField')(default=0)),
+        ))
+        db.send_create_signal('super_archives', ['MailingList'])
+
+        # Adding model 'MailingListMembership'
+        db.create_table('super_archives_mailinglistmembership', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('user', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['auth.User'])),
+            ('mailinglist', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.MailingList'])),
+        ))
+        db.send_create_signal('super_archives', ['MailingListMembership'])
+
+        # Adding model 'Thread'
+        db.create_table('super_archives_thread', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('subject_token', self.gf('django.db.models.fields.CharField')(max_length=512)),
+            ('mailinglist', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.MailingList'])),
+            ('latest_message', self.gf('django.db.models.fields.related.OneToOneField')(related_name='+', unique=True, null=True, to=orm['super_archives.Message'])),
+        ))
+        db.send_create_signal('super_archives', ['Thread'])
+
+        # Adding unique constraint on 'Thread', fields ['subject_token', 'mailinglist']
+        db.create_unique('super_archives_thread', ['subject_token', 'mailinglist_id'])
+
+        # Adding model 'Vote'
+        db.create_table('super_archives_vote', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('user', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['auth.User'])),
+            ('message', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.Message'])),
+        ))
+        db.send_create_signal('super_archives', ['Vote'])
+
+        # Adding unique constraint on 'Vote', fields ['user', 'message']
+        db.create_unique('super_archives_vote', ['user_id', 'message_id'])
+
+        # Adding model 'Message'
+        db.create_table('super_archives_message', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('from_address', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.EmailAddress'])),
+            ('mailinglist', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.MailingList'])),
+            ('thread', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.Thread'], null=True)),
+            ('subject', self.gf('django.db.models.fields.CharField')(max_length=512)),
+            ('subject_clean', self.gf('django.db.models.fields.CharField')(max_length=512)),
+            ('body', self.gf('django.db.models.fields.TextField')(default='')),
+            ('received_time', self.gf('django.db.models.fields.DateTimeField')()),
+            ('message_id', self.gf('django.db.models.fields.CharField')(max_length=512)),
+        ))
+        db.send_create_signal('super_archives', ['Message'])
+
+        # Adding model 'MessageMetadata'
+        db.create_table('super_archives_messagemetadata', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('Message', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['super_archives.Message'])),
+            ('name', self.gf('django.db.models.fields.CharField')(max_length=512)),
+            ('value', self.gf('django.db.models.fields.TextField')()),
+        ))
+        db.send_create_signal('super_archives', ['MessageMetadata'])
+
+
+    def backwards(self, orm):
+        
+        # Removing unique constraint on 'Vote', fields ['user', 'message']
+        db.delete_unique('super_archives_vote', ['user_id', 'message_id'])
+
+        # Removing unique constraint on 'Thread', fields ['subject_token', 'mailinglist']
+        db.delete_unique('super_archives_thread', ['subject_token', 'mailinglist_id'])
+
+        # Deleting model 'PageHit'
+        db.delete_table('super_archives_pagehit')
+
+        # Deleting model 'EmailAddress'
+        db.delete_table('super_archives_emailaddress')
+
+        # Deleting model 'UserProfile'
+        db.delete_table('super_archives_userprofile')
+
+        # Deleting model 'MailingList'
+        db.delete_table('super_archives_mailinglist')
+
+        # Deleting model 'MailingListMembership'
+        db.delete_table('super_archives_mailinglistmembership')
+
+        # Deleting model 'Thread'
+        db.delete_table('super_archives_thread')
+
+        # Deleting model 'Vote'
+        db.delete_table('super_archives_vote')
+
+        # Deleting model 'Message'
+        db.delete_table('super_archives_message')
+
+        # Deleting model 'MessageMetadata'
+        db.delete_table('super_archives_messagemetadata')
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,133 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'UserProfile.verification_hash'
+        db.add_column('super_archives_userprofile', 'verification_hash', self.gf('django.db.models.fields.CharField')(max_length=32, null=True), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'UserProfile.verification_hash'
+        db.delete_column('super_archives_userprofile', 'verification_hash')
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,134 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'Thread.score'
+        db.add_column('super_archives_thread', 'score', self.gf('django.db.models.fields.IntegerField')(default=0), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'Thread.score'
+        db.delete_column('super_archives_thread', 'score')
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'score': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,135 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'Vote.created'
+        db.add_column('super_archives_vote', 'created', self.gf('django.db.models.fields.DateTimeField')(auto_now_add=True, default=datetime.datetime(2012, 1, 19, 18, 8, 46, 813949), blank=True), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'Vote.created'
+        db.delete_column('super_archives_vote', 'created')
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'score': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,143 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'Message.spam'
+        db.add_column('super_archives_message', 'spam', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False)
+
+        # Adding field 'Thread.spam'
+        db.add_column('super_archives_thread', 'spam', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'Message.spam'
+        db.delete_column('super_archives_message', 'spam')
+
+        # Deleting field 'Thread.spam'
+        db.delete_column('super_archives_thread', 'spam')
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'score': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,143 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding index on 'Message', fields ['subject_clean']
+        db.create_index('super_archives_message', ['subject_clean'])
+
+        # Adding index on 'Message', fields ['subject']
+        db.create_index('super_archives_message', ['subject'])
+
+
+    def backwards(self, orm):
+        
+        # Removing index on 'Message', fields ['subject']
+        db.delete_index('super_archives_message', ['subject'])
+
+        # Removing index on 'Message', fields ['subject_clean']
+        db.delete_index('super_archives_message', ['subject_clean'])
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512', 'db_index': 'True'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512', 'db_index': 'True'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'score': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
@@ -0,0 +1,137 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding index on 'EmailAddress', fields ['real_name']
+        db.create_index('super_archives_emailaddress', ['real_name'])
+
+
+    def backwards(self, orm):
+        
+        # Removing index on 'EmailAddress', fields ['real_name']
+        db.delete_index('super_archives_emailaddress', ['real_name'])
+
+
+    models = {
+        'auth.group': {
+            'Meta': {'object_name': 'Group'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '80'}),
+            'permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'})
+        },
+        'auth.permission': {
+            'Meta': {'ordering': "('content_type__app_label', 'content_type__model', 'codename')", 'unique_together': "(('content_type', 'codename'),)", 'object_name': 'Permission'},
+            'codename': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'content_type': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['contenttypes.ContentType']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'})
+        },
+        'auth.user': {
+            'Meta': {'object_name': 'User'},
+            'date_joined': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'blank': 'True'}),
+            'first_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Group']", 'symmetrical': 'False', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_active': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'is_staff': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_superuser': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'last_login': ('django.db.models.fields.DateTimeField', [], {'default': 'datetime.datetime.now'}),
+            'last_name': ('django.db.models.fields.CharField', [], {'max_length': '30', 'blank': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'user_permissions': ('django.db.models.fields.related.ManyToManyField', [], {'to': "orm['auth.Permission']", 'symmetrical': 'False', 'blank': 'True'}),
+            'username': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '30'})
+        },
+        'contenttypes.contenttype': {
+            'Meta': {'ordering': "('name',)", 'unique_together': "(('app_label', 'model'),)", 'object_name': 'ContentType', 'db_table': "'django_content_type'"},
+            'app_label': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'model': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'super_archives.emailaddress': {
+            'Meta': {'object_name': 'EmailAddress'},
+            'address': ('django.db.models.fields.EmailField', [], {'unique': 'True', 'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'md5': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'real_name': ('django.db.models.fields.CharField', [], {'db_index': 'True', 'max_length': '64', 'blank': 'True'}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'emails'", 'null': 'True', 'to': "orm['auth.User']"})
+        },
+        'super_archives.mailinglist': {
+            'Meta': {'object_name': 'MailingList'},
+            'description': ('django.db.models.fields.TextField', [], {}),
+            'email': ('django.db.models.fields.EmailField', [], {'max_length': '75'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'last_imported_index': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logo': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '80'})
+        },
+        'super_archives.mailinglistmembership': {
+            'Meta': {'object_name': 'MailingListMembership'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        },
+        'super_archives.message': {
+            'Meta': {'object_name': 'Message'},
+            'body': ('django.db.models.fields.TextField', [], {'default': "''"}),
+            'from_address': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.EmailAddress']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'message_id': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'received_time': ('django.db.models.fields.DateTimeField', [], {}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject': ('django.db.models.fields.CharField', [], {'max_length': '512', 'db_index': 'True'}),
+            'subject_clean': ('django.db.models.fields.CharField', [], {'max_length': '512', 'db_index': 'True'}),
+            'thread': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Thread']", 'null': 'True'})
+        },
+        'super_archives.messagemetadata': {
+            'Message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'Meta': {'object_name': 'MessageMetadata'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '512'}),
+            'value': ('django.db.models.fields.TextField', [], {})
+        },
+        'super_archives.pagehit': {
+            'Meta': {'object_name': 'PageHit'},
+            'hit_count': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'url_path': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '2048', 'db_index': 'True'})
+        },
+        'super_archives.thread': {
+            'Meta': {'unique_together': "(('subject_token', 'mailinglist'),)", 'object_name': 'Thread'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'latest_message': ('django.db.models.fields.related.OneToOneField', [], {'related_name': "'+'", 'unique': 'True', 'null': 'True', 'to': "orm['super_archives.Message']"}),
+            'mailinglist': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.MailingList']"}),
+            'score': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'spam': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'subject_token': ('django.db.models.fields.CharField', [], {'max_length': '512'})
+        },
+        'super_archives.userprofile': {
+            'Meta': {'object_name': 'UserProfile'},
+            'facebook': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'google_talk': ('django.db.models.fields.EmailField', [], {'max_length': '75', 'null': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'institution': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'role': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'twitter': ('django.db.models.fields.CharField', [], {'max_length': '128', 'null': 'True'}),
+            'user': ('django.db.models.fields.related.OneToOneField', [], {'to': "orm['auth.User']", 'unique': 'True'}),
+            'verification_hash': ('django.db.models.fields.CharField', [], {'max_length': '32', 'null': 'True'}),
+            'webpage': ('django.db.models.fields.CharField', [], {'max_length': '256'})
+        },
+        'super_archives.vote': {
+            'Meta': {'unique_together': "(('user', 'message'),)", 'object_name': 'Vote'},
+            'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'message': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['super_archives.Message']"}),
+            'user': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['auth.User']"})
+        }
+    }
+
+    complete_apps = ['super_archives']
-# -*- coding: utf8 -*-
+# -*- coding: utf-8 -*-
  
+import datetime
 from hashlib import md5
  
 from django.db import models
 from django.conf import settings
 from django.contrib.auth.models import User
-from django.core.urlresolvers import reverse
+from django.core.urlresolvers import reverse, NoReverseMatch
+
+
+class NotSpamManager(models.Manager):
+    """Only return objects which are not marked as spam."""
+
+    def get_query_set(self):
+        return super(NotSpamManager, self).get_query_set().exclude(spam=True)
+
+
+class PageHit(models.Model):
+    url_path = models.CharField(max_length=2048, unique=True, db_index=True)
+    hit_count = models.IntegerField(default=0)
  
  
 class EmailAddress(models.Model):
     user = models.ForeignKey(User, null=True, related_name='emails') 
     address = models.EmailField(unique=True)
-    real_name = models.CharField(max_length=64, blank=True)
+    real_name = models.CharField(max_length=64, blank=True, db_index=True)
     md5 = models.CharField(max_length=32, null=True)
  
     def save(self, *args, **kwargs):
@@ -28,12 +41,14 @@ class EmailAddress(models.Model):
  
     def get_profile_link(self):
         if self.user:
-            return reverse('colab.views.user_profile_username',
-                           args=[self.user.username])
+            return reverse('user_profile', args=[self.user.username])
         else:
-            return reverse('colab.views.user_profile_emailhash',
+            return reverse('colab.views.userprofile.by_emailhash',
                            args=[self.md5])
  
+    def __unicode__(self):
+        return '"%s" <%s>' % (self.get_full_name(), self.address)
+
  
 class UserProfile(models.Model):
     user = models.OneToOneField(User, unique=True)
@@ -43,6 +58,10 @@ class UserProfile(models.Model):
     facebook = models.CharField(max_length=128, null=True)
     google_talk = models.EmailField(null=True)
     webpage = models.CharField(max_length=256)
+    verification_hash = models.CharField(max_length=32, null=True)
+
+    def __unicode__(self):
+        return '%s (%s)' % (self.user.get_full_name(), self.user.username)
  
 # This does the same the same than related_name argument but it also creates
 #   a profile in the case it doesn't exist yet. 
@@ -69,17 +88,80 @@ class MailingListMembership(models.Model):
  
  
 class Thread(models.Model):
+    class Meta:
+        unique_together = ('subject_token', 'mailinglist')
+    
     subject_token = models.CharField(max_length=512)
     mailinglist = models.ForeignKey(MailingList)
     latest_message = models.OneToOneField('Message', null=True, 
-                                                     related_name='+')           
-    class Meta:
-        unique_together = ('subject_token', 'mailinglist')
+                                                     related_name='+')
+    score = models.IntegerField(default=0)
+    spam = models.BooleanField(default=False)
+    
+    all_objects = models.Manager()
+    objects = NotSpamManager()
+
+    def __unicode__(self):
+        return '%s - %s (%s)' % (self.id,
+                                 self.subject_token, 
+                                 self.message_set.count())
+
+    def update_score(self):
+        """Update the relevance score for this thread.
+        
+        The score is calculated with the following variables:
+
+        * vote_weight: 100 - (minus) 1 for each 3 days since 
+          voted with minimum of 5.
+        * replies_weight: 300 - (minus) 1 for each 3 days since 
+          replied with minimum of 5.
+        * page_view_weight: 10.
+
+        * vote_score: sum(vote_weight)
+        * replies_score: sum(replies_weight)
+        * page_view_score: sum(page_view_weight)
+
+        * score = (vote_score + replies_score + page_view_score) // 10
+        with minimum of 0 and maximum of 5000
+
+        """
+
+        if not self.subject_token:
+            return    
+ 
+        # Save this pseudo now to avoid calling the
+        #   function N times in the loops below
+        now = datetime.datetime.now()
+        days_ago = lambda date: (now - date).days
+        get_score = lambda weight, created: \
+                                  max(weight - (days_ago(created) // 3), 5)
+
+        vote_score = 0
+        replies_score = 0
+        for msg in self.message_set.all():
+            # Calculate replies_score
+            replies_score += get_score(300, msg.received_time)
+
+            # Calculate vote_score
+            for vote in msg.vote_set.all():
+                vote_score += get_score(100, vote.created)
+
+        # Calculate page_view_score       
+        try: 
+            url = reverse('thread_view', args=[self.subject_token])
+            pagehit = PageHit.objects.get(url_path=url)
+            page_view_score = pagehit.hit_count * 10
+        except (NoReverseMatch, PageHit.DoesNotExist):
+            page_view_score = 0
+
+        self.score = (page_view_score + vote_score + replies_score) // 10
+        self.save()
  
  
 class Vote(models.Model):
     user = models.ForeignKey(User)
     message = models.ForeignKey('Message')
+    created = models.DateTimeField(auto_now_add=True)
  
     class Meta:
         unique_together = ('user', 'message')
@@ -91,20 +173,26 @@ class Vote(models.Model):
  
 class Message(models.Model):
  
-    from_address = models.ForeignKey(EmailAddress)
+    from_address = models.ForeignKey(EmailAddress, db_index=True)
     mailinglist = models.ForeignKey(MailingList)
-    thread = models.ForeignKey(Thread, null=True)
+    thread = models.ForeignKey(Thread, null=True, db_index=True)
     # RFC 2822 recommends to use 78 chars + CRLF (so 80 chars) for
     #   the max_length of a subject but most of implementations
     #   goes for 256. We use 512 just in case.
-    subject = models.CharField(max_length=512)
-    subject_clean = models.CharField(max_length=512)
+    subject = models.CharField(max_length=512, db_index=True)
+    subject_clean = models.CharField(max_length=512, db_index=True)
     body = models.TextField(default='')
     received_time = models.DateTimeField()
     message_id = models.CharField(max_length=512)
+    spam = models.BooleanField(default=False)
  
+    all_objects = models.Manager()
+    objects = NotSpamManager()
+    
     def __unicode__(self):
-        return 'Email Message Id: %s' % self.id
+        return '(%s) %s: %s' % (self.id, 
+                                self.from_address.get_full_name(), 
+                                self.subject_clean)
  
     def vote_list(self):
         """Return a list of user that voted in this message."""
@@ -125,7 +213,27 @@ class Message(models.Model):
             message=self,
             user=user
         ).delete()
-        
+
+    @property
+    def url(self):
+        """Shortcut to get thread url"""
+        return reverse('thread_view', args=[self.thread.subject_token])
+    
+    @property
+    def Description(self):
+        """Alias to self.body"""
+        return self.body
+
+    @property
+    def Title(self):
+        """Alias to self.subject_clean"""
+        return self.subject_clean
+
+    @property
+    def modified(self):
+        """Alias to self.modified"""
+        return self.received_time
+
  
 class MessageMetadata(models.Model):
     Message = models.ForeignKey(Message)
@@ -137,4 +245,4 @@ class MessageMetadata(models.Model):
     def __unicode__(self):
         return 'Email Message Id: %s - %s: %s' % (self.Message.id,
                                                   self.name, self.value)
-
+    
  
-from super_archives.models import Thread, Vote, Message
+from colab.super_archives.models import Thread, Vote, Message, PageHit
  
  
 def get_messages_by_date():
@@ -34,30 +34,14 @@ def get_latest_threads():
     return Thread.objects.order_by('-latest_message__received_time')
  
  
-def get_voted_threads():
-    """Query for the most voted threads sorting by the sum of votes 
-    and latest messages received.
-    
-    NOTE: This implementation has serious performance issues on
-    MySQL databases but it performes quite well on PostgreSQL.
-    
-    """
-    
-    sql = """
-        SELECT 
-            count(sav.id)
-        FROM
-            super_archives_message AS sam 
-            JOIN super_archives_vote AS sav
-                ON sav.message_id = sam.id
-        WHERE
-            super_archives_thread.id = sam.thread_id
-    """
+def get_hotest_threads():
+    return Thread.objects.order_by('-score', '-latest_message__received_time')
  
-    threads = Thread.objects.extra(
-        select={
-            'vote_count': sql
-        }
-    )
-    return threads.order_by('-vote_count', '-latest_message__received_time')
+
+def get_page_hits(path_info):
+    pagehit = PageHit.objects.filter(url_path=path_info)
  
+    if pagehit:
+        return pagehit[0].hit_count
+    return 0
+
 {% extends "base.html" %}
+{% load i18n %}
 {% load append_to_get %}
 {% block main-content %}
 <div id="message-list">
@@ -10,27 +11,30 @@
  
     <h4>Ordenar por</h4>
     <ul>
-      <li><a href="{% append_to_get order='latest',p=1 %}">Relevância</a></li>
-      <li><a href="{% append_to_get order='latest',p=1 %}">Atividade recente</a></li>
-      <li><a href="{% append_to_get order='voted',p=1 %}">Mais votadas</a></li>
+      <li {% ifequal order_by "hotest" %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+        <a href="{% ifequal order_by "hotest" %} {% append_to_get order="",p=1 %} {% else %} {% append_to_get order='hotest',p=1 %} {% endifequal %}">
+          Relevância</a></li>
+      <li {% ifequal order_by "latest" %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+        <a href="{% ifequal order_by "latest" %} {% append_to_get order="",p=1 %} {% else %} {% append_to_get order='latest',p=1 %} {% endifequal %}">
+          Atividade recente</a></li>
     </ul>
  
     <hr class="space" />
  
     <h4>Listas</h4>
     <ul>
-    {% for list in lists %}
-      <li><a href="{% append_to_get list=list.name,p=1 %}">{{ list.name }}</a></li>
-    {% endfor %}
+      {% for list in lists %}
+        <li {% ifequal list.name selected_list %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+          <a href="{% ifnotequal list.name selected_list %} {% append_to_get list=list.name,p=1 %} {% else %} {% append_to_get list="",p=1 %}
+          {% endifnotequal %}">{{ list.name }}</a></li>
+      {% endfor %}
     </ul>
   </div>
  
   <div class="span-17 prepend-1 last">
     <ul>
       {% for thread in threads.object_list %}
-        {% with thread.latest_message as email %}
-          {% include "message-preview.html" %}
-        {% endwith %}
+        {% include "message-preview.html" with doc=thread.latest_message %}
       {% empty %}
         <br/><br/>
         <span class="span-18 center large">
@@ -61,4 +65,4 @@
   </div>
 </div>
  
-{% endblock %}
 \ No newline at end of file
+{% endblock %}
 {% load i18n %}
-{% with email.from_address as from %}
-{% if email.subject_clean %}
+
+{% if doc.Title %}
 <li class="preview-message">
-  <a href="{% url super_archives.views.list_messages %}?list={{ email.thread.mailinglist.name }}">
-    <div class="left">
-      <span class="tag">{{ email.mailinglist }}</span>
-    </div>
+  {% if doc.Type %}
+  <img alt="{{ doc.Type }}" title="{{ doc.Type }}" 
+       src="{{ STATIC_URL }}img/{{ doc.Type }}.png" />
+  {% endif %}
+  
+  {% if doc.mailinglist %}
+  <a href="{% url super_archives.views.list_messages %}?list={{ doc.mailinglist }}">
+    <span class="tag">{{ doc.mailinglist }}</span>
   </a>
-  <div class="subject">
-    <a href="{% url super_archives.views.thread email.thread.subject_token %}">
-      {{ email.subject_clean }}
+  {% endif %}
+
+  <span class="subject">
+    <a href="{{ doc.url }}"
+       {% if 
+       title="{% filter striptags|truncatewords:50 %}
+                {{ doc.Description|escape }}
+              {% endfilter %}">
+      {{ doc.Title }}
     </a>
-  </div>
+  </span>
+  
+  <span class="quiet">- 
+    {{ doc.Description|striptags }}
+  </span>
+  
   <div class="quiet">
     <span class="left">
-      {% trans "por" %}
-      <a href="{{ from.get_profile_link }}">
-        {% firstof from.get_full_name "Anônimo" %}
+      {% trans "criado por" %}
+      {% if doc.from_address.get_full_name %}
+      <a href="{{ doc.from_address.get_profile_link }}">
+        {{ doc.from_address.get_full_name }}
       </a>
-    </span>          
+      {% else %}
+        {% firstof doc.Creator _("anônimo") %}
+      {% endif %}
+    </span>    
+          
     <span class="right">
-      {{ email.received_time|timesince }} {% trans "atrás" %} 
+      {{ doc.modified|timesince }}
+      {% trans "atrás" %} 
     </span>
   </div>
 </li>
 {% endif %}
-{% endwith %}
 \ No newline at end of file
@@ -2,9 +2,11 @@
 {% load i18n %}
 {% load append_to_get %}
 {% block main-content %}
+  <div id="vote-notification" class="error hide"></div>
+
   <h2>{{ first_msg.subject_clean }}</h2>
   <hr/>
-  <div class="span-19 border">
+  <div class="span-17 border">
     <ul>
     {% for email in emails %}
       <li>
@@ -21,7 +23,7 @@
  
             <div class="plus">
               <span>{{ email.votes_count }}</span>
-              <img class="right" src="{{ STATIC_URL }}img/thumbs_up.jpg">
+              <img title="{% trans 'Votar' %}" class="right" src="{{ STATIC_URL }}img/plus.png">
             </div>
  
             <p class="minus {% if not user in email.vote_list %}hide{% endif %}">
@@ -29,7 +31,7 @@
             </p>
           </div>
  
-          <div class="span-15">
+          <div class="span-13">
             <pre>{{ email.body }}</pre>
           </div>
         </div>
@@ -40,11 +42,36 @@
     {% endfor %}
     </ul>
   </div>
-  <div class="span-4 filters last">
-    <h4>Ordenar por</h4>
+  <div class="span-6 filters last">
+    <h4><b>{% trans "Ordenar por" %}:</b></h4>
+    <ul>
+      <li><a href="{% append_to_get order='voted' %}">{% trans "Votos" %}</a></li>
+      <li><a href="{% append_to_get order='date' %}">{% trans "Data" %}</a></li>
+    </ul>
+
+    <hr class="space"/>
+    
+    <h4><b>{% trans "Estatísticas:" %}</b></h4>
+    
     <ul>
-      <li><a href="{% append_to_get order='voted' %}">Votos</a></li>
-      <li><a href="{% append_to_get order='date' %}">Data</a></li>
+      <li class="quiet">{% trans "iniciada à" %}
+        <h4>{{ first_msg.received_time|timesince }} {% trans "atrás" %}</h4>
+      </li>
+      <li class="quiet">{% trans "visualizada" %}
+        <h4>{{ pagehits }} {% trans "vezes" %}</h4>
+      </li>
+      <li class="quiet">{% trans "respondida" %}
+        <h4>{{ emails|length }} {% trans "vezes" %}</h4>
+      </li>
+      <li class="quiet">{% trans "votada" %}
+        <h4>{{ total_votes }} {% trans "vezes" %}</h4>
+      </li>
     </ul>
+    
   </div>
-{% endblock %}  
 \ No newline at end of file
+  
+  <script type="text/javascript" charset="utf-8">
+    pagehit("{{ request.path_info }}");
+  </script>
+  
+{% endblock %}  
+
+import urllib
 from django import template
  
 register = template.Library()
@@ -33,15 +35,10 @@ class AppendGetNode(template.Node):
             get[key] = self.dict_pairs[key].resolve(context)
  
         path = context['request'].META['PATH_INFO']
-        
-        #print "&".join(["%s=%s" % (key, value) 
-        #                for (key, value) in get.items() 
-        #                if value])
-        
+                
         if len(get):
-            path += "?%s" % "&".join(["%s=%s" % (key, value) 
-                                      for (key, value) in get.items() 
-                                      if value])
+            path = '?' + urllib.urlencode(get)
+
         return path
  
 @register.tag()
 from django import template
+from django import forms
  
 def render_form_field(parser, token):
     variables = token.split_contents()
@@ -9,7 +10,7 @@ def render_form_field(parser, token):
     elif len(variables) == 3:
         tag_name, form_field, default_value = variables
     else:
-        raise TemplateSyntaxError
+        raise template.TemplateSyntaxError
  
     return RenderFormField(form_field, default_value)
  
@@ -26,7 +27,10 @@ class RenderFormField(template.Node):
         class_ = ''
         errors = ''
         form_field_tag = ''
-        form_field = self.form_field_nocontext.resolve(context)
+        try:
+            form_field = self.form_field_nocontext.resolve(context)
+        except template.VariableDoesNotExist:
+            return ''
  
         if form_field.errors:
             class_ += 'error'
@@ -42,6 +46,9 @@ class RenderFormField(template.Node):
  
         if editable:
             form_field_tag = '<br/>' + str(form_field)
+        elif isinstance(form_field.field, forms.URLField):
+            form_field_tag = """<a href="%s" target="_blank">%s</a>""" % (
+                default_value, default_value)
         else:
             form_field_tag = default_value
  
@@ -53,4 +60,4 @@ class RenderFormField(template.Node):
         )
  
 register = template.Library()
-register.tag('render_form_field', render_form_field)
 \ No newline at end of file
+register.tag('render_form_field', render_form_field)
@@ -2,6 +2,8 @@ from django.conf.urls.defaults import patterns, include, url
  
 urlpatterns = patterns('',
 #    url(r'thread/(?P<thread>\d+)/$', 'super_archives.views.thread', name='thread'),
-    url(r'thread/(?P<thread_token>[-\w]+)$', 'super_archives.views.thread'),
-    url(r'thread/$', 'super_archives.views.list_messages')
+    url(r'thread/(?P<thread_token>[-\w]+)$', 
+        'colab.super_archives.views.thread', name="thread_view"),
+    url(r'thread/$', 
+        'colab.super_archives.views.list_messages', name='thread_list')
 )
-# -*- coding: utf8 -*-
+# -*- coding: utf-8 -*-
  
 from django.core.exceptions import ValidationError
  
-# -*- coding: utf8 -*-
+# -*- coding: utf-8 -*-
  
 from django.template import RequestContext
 from django.core.paginator import Paginator
 from django.shortcuts import render_to_response, get_list_or_404
  
-from super_archives import queries
-from super_archives.models import MailingList
+from colab.super_archives import queries
+from colab.super_archives.models import MailingList, Thread
  
  
 def thread(request, thread_token):
  
     first_message = queries.get_first_message_in_thread(thread_token)
     order_by = request.GET.get('order')
-    if order_by == 'date':
-        msgs_query = queries.get_messages_by_date()
-    else:
+    if order_by == 'voted':
         msgs_query = queries.get_messages_by_voted()
+    else:
+        msgs_query = queries.get_messages_by_date()
  
     msgs_query = msgs_query.filter(thread__subject_token=thread_token)
     emails = msgs_query.exclude(id=first_message.id)
+    
+    total_votes = first_message.votes_count()
+    for email in emails:
+        total_votes += email.votes_count()
+
+    # Update relevance score
+    thread = Thread.objects.get(subject_token=thread_token)   
+    thread.update_score()
+ 
     template_data = {
-        'request': request,
         'first_msg': first_message,
         'emails': [first_message] + list(emails),
+        'pagehits': queries.get_page_hits(request.path_info),
+        'total_votes': total_votes,
     }
+    
     return render_to_response('message-thread.html', template_data, 
                               RequestContext(request))
  
  
 def list_messages(request):
  
+    selected_list = request.GET.get('list')
+
     order_by = request.GET.get('order')
-    if order_by == 'voted':
-        threads = queries.get_voted_threads()
+    if order_by == 'hotest':
+        threads = queries.get_hotest_threads()
     else:
         threads = queries.get_latest_threads()
  
@@ -50,7 +63,8 @@ def list_messages(request):
         'lists': lists,
         'n_results': paginator.count,
         'threads': threads,
-        'request': request,
+        'selected_list': selected_list,
+        'order_by': order_by,
     }
     return render_to_response('message-list.html', template_data, 
-                              RequestContext(request))
 \ No newline at end of file
+                              RequestContext(request))
@@ -0,0 +1 @@
+<h1>Not found. Keep searching! :)</h1>
 \ No newline at end of file
@@ -0,0 +1 @@
+<h1>Ooopz... something went wrong!</h1>
 \ No newline at end of file
@@ -0,0 +1,24 @@
+{% extends "base.html" %}
+{% load i18n %}
+{% load form_field %}
+
+{% block main-content %}
+
+{% if form.errors %}
+  <div class="alert">
+    <b>Por favor, corrija os erros abaixo e tente novamente.</b>
+  </div>
+{% endif %}
+
+<form method="POST" action=".">
+  {% csrf_token %}
+
+  <fieldset class="span-24 box">
+    {% render_form_field form.old_password %}
+    {% render_form_field form.new_password1 %}
+    {% render_form_field form.new_password2 %}
+    <input type="submit" value="{% trans 'Alterar senha' %}"/>
+  </fieldset>
+  
+</form>
+{% endblock %}
 \ No newline at end of file
@@ -0,0 +1,10 @@
+{% extends "base.html" %}
+{% load i18n %}
+
+{% block main-content %}
+
+<span class="span-24 {{ msg_css_class }}">
+  {% trans msg %}
+</span>
+
+{% endblock %}
 \ No newline at end of file
@@ -0,0 +1,18 @@
+{% extends "base.html" %}
+{% load i18n %}
+
+{% block main-content %}
+<h2>{% trans "Esqueci minha senha" %}</h2>
+
+<form method="POST" action="{% url request_reset_password %}">
+  {% csrf_token %}
+
+  <fieldset class="span-24 center box">
+    <label>{% trans "Usuário" %}:</label>
+    <input name="username"/>
+    <input type="submit" value="{% trans 'Enviar nova senha' %}"/>
+  </fieldset>
+
+</form>
+
+{% endblock %}
 \ No newline at end of file
+{% load i18n %}
 <html>
   <head>
-    <!-- Framework CSS -->  
-    <link rel="stylesheet" href="{{ STATIC_URL }}css/blueprint/screen.css" 
-                                  type="text/css" media="screen, projection" />  
-    <link rel="stylesheet" href="{{ STATIC_URL }}css/blueprint/print.css" 
-                                  type="text/css" media="print" />  
-    <!--[if IE]><link rel="stylesheet" 
-                      href="{{ STATIC_URL }}css/blueprint/ie.css" 
-                      type="text/css" media="screen, projection" /><![endif]-->  
-
-    <link rel="stylesheet" href="{{ STATIC_URL }}css/screen.css" 
+    <!-- Framework CSS -->
+    <link rel="stylesheet" href="{{ STATIC_URL }}css/blueprint/screen.css"
+                                  type="text/css" media="screen, projection" />
+    <link rel="stylesheet" href="{{ STATIC_URL }}css/blueprint/print.css"
+                                  type="text/css" media="print" />
+    <!--[if IE]><link rel="stylesheet"
+                      href="{{ STATIC_URL }}css/blueprint/ie.css"
+                      type="text/css" media="screen, projection" /><![endif]-->
+
+    <link rel="stylesheet" href="{{ STATIC_URL }}css/screen.css"
                                 type="text/css" media="screen" charset="utf-8"/>
  
     <script type="text/javascript" src="{{ STATIC_URL }}js/jquery-1.7.min.js"></script>    
     <script type="text/javascript" src="{{ STATIC_URL }}js/base.js"></script>
-    
+
     {% block head_js %}
     {% endblock %}
+    {% block head_css %}
+      <!-- css used to show the selected item on filters -->
+      <style type="text/css">
+        .selected {
+            font-weight: bold;
+        }
+
+        .selected a {
+            text-decoration: none;
+            color: #000;
+            background-image: url('{{ STATIC_URL }}img/x.png');
+            background-repeat: no-repeat;
+            background-position: 0 3px;
+            padding-left: 16px;
+        }
+
+        .selected a:hover {
+          text-decoration: underline;
+        }
+
+        li.selected:before {
+            content: '';
+        }
+      </style>
+
+    {% endblock %}
   </head>
  
   <body class="container">
@@ -24,55 +51,66 @@
       <div id="top-menu" class="right">
         {% if not user.is_authenticated %}
         <span class="colborder">
-          <a href="{% url colab.views.signup %}">Cadastre-se</a>
+          <a href="{% url signup %}">{% trans "Cadastre-se" %}</a>
         </span>
         <span>
-          <a href="{% url django.contrib.auth.views.login %}">Acessar</a>
+          <a href="{% url login %}?next={{ request.path }}">{% trans "Acessar" %}</a>
         </span>
         {% else %}
         <span class="colborder">
-          <a href="{% url colab.views.user_profile_username user.username %}">Meu Perfil</a>
+          {% trans "autenticado como" %} <b>{{ user.username }}</b>
+        </span>
+        <span class="colborder">
+          <a href="{% url user_profile user.username %}">
+            {% trans "Meu Perfil" %}
+          </a>
         </span>
         <span>
-          <a href="{% url django.contrib.auth.views.logout %}">Sair</a>
+          <a href="{% url logout %}">
+            {% trans "Sair" %}
+          </a>
         </span>
         {% endif %}
       </div>
-      
+
       {% block header %}
         <h1><span class="hide">COLAB</span>
           <a href="/">
             <img src="{{ STATIC_URL }}img/logo_small.png" alt="Colab" /></a>
         </h1>
       {% endblock %}
-      
-        <div id="header-menu" class="span-24">
+
+      <div id="header-menu" class="span-24">
  
         <span class="colborder">
-          <a href="#">Iniciar Discussão</a>
-        </span>
-        <span class="colborder">
-          <a href="{% url super_archives.views.list_messages %}">Discussões</a>
+          <a href="{% url thread_list %}">Discussões</a>
         </span>
         <span class="colborder">
-          <a href="http://colab.interlegis.gov.br/wiki">Wiki</a>
+          <a href="http://colab.interlegis.gov.br/wiki" target="_blank">Wiki</a>
         </span>
         <span class="colborder">
-          <a href="#">Contribua</a>
+          <a href="http://listas.interlegis.gov.br/mailman/listinfo/" 
+             target="_blank">Contribua</a>
         </span>
         <span class="colborder">
-          <a href="#">Reporte um problema</a>
+          <a href="http://colab.interlegis.leg.br/newticket"
+             target="_blank">Reporte um problema</a>
         </span>
-        <span class="colborder">
-          <a href="#">Treco</a>
+        <span>
+          <a href="#">Mensageiro</a>
         </span>
  
-        <input id="header-searchbox" type="text" placeholder="Pesquise aqui..."/>
-        <input type="button" value="Buscar"></input>
+        <span class="right">
+          <form action="/search/" method="GET">
+            <input name="q" id="header-searchbox" value="{{ request.GET.q }}"
+                   type="text" placeholder="Pesquise aqui..." />
+            <input type="submit" value="Buscar"></input>
+          </form>
+        </span>
       </div>
     </div>
-    
-    <hr/> 
+
+    <hr/>
  
     <div id="main-content" class="span-24">
       {% block main-content %} {% endblock %}
@@ -82,11 +120,11 @@
  
     <div id="footer" class="span-24 center">
       {% block footer %}
-        <p>O conteúdo deste site está publicado sob a licença <a 
-          href="http://creativecommons.org/licenses/by-nc-sa/2.0/br/">Creative 
+        <p>O conteúdo deste site está publicado sob a licença <a
+          href="http://creativecommons.org/licenses/by-nc-sa/2.0/br/">Creative
           Commons - atribuição e não-comercial</a>
         </p>
       {% endblock %}
     </div>
   </body>
-</html>
 \ No newline at end of file
+</html>
@@ -0,0 +1,19 @@
+{% load i18n %}
+
+{% blocktrans %}
+  Este email foi enviado para confirmar a solicitação de troca de senha 
+  para o usuário <b>{{ username }}</b> do Colab Interlegis. Caso você não 
+  tenha realizado a solicitação por favor ignore esta mensagem.
+{% endblocktrans %}
+
+<br/>
+<br/>
+
+{% blocktrans %}
+  Para realizar a troca de senha acesse o link abaixo:
+{% endblocktrans %}
+<br/>
+
+<a href="http://{{ server_name }}{% url reset_password hash %}"> 
+  http://{{ server_name }}{% url reset_password hash %}
+</a>
 \ No newline at end of file
@@ -0,0 +1,9 @@
+{% load i18n %}
+
+Bem-vindo ao Colab!
+
+Para ativar sua conta por favor valide seu email acessando o link seguinte:
+
+<a href="http://{{ server_name }}{% url email_verification hash %}"> 
+  http://{{ server_name }}{% url email_verification hash %}
+</a>
 \ No newline at end of file
 {% extends "base.html" %}
 {% load i18n %}
  
-{% block header %}      
+{% block head_js %}
+  {% include "pizza-chart.html" with chart_div="collabs" chart_width=410 chart_height=330 %}
+{% endblock %}
+
+{% block header %}
   <div id="header-logo" class="span-24">
     <img class="center" src="{{ STATIC_URL }}img/logo.png" />
   </div>
@@ -12,34 +16,52 @@
 {% endblock %}
  
 {% block main-content %}
-  <h3 class="span-13">{% trans "Últimas Menssagens:" %}</h3>
-  <h3 class="span-11 last">{% trans "Discussões mais votadas:" %}</h3>
-  
-  <ul id="latest_emails" class="span-12 colborder">
-    {% for thread in latest_threads %}
-    {% with thread.latest_message as email %}
-      {% include "message-preview.html" %}
-    {% endwith %}
-    {% endfor %}
-  </ul>
-
-  <ul id="hotest_emails" class="span-11 last">
-    {% for thread in hotest_threads %}
-    {% with thread.latest_message as email %}
-      {% include "message-preview.html" %}
-    {% endwith %}
-    {% endfor %}
-  </ul>
-  
+
+  <div class="span-12 colborder">
+    <h3>{% trans "Últimas Colaborações:" %}</h3>
+    <ul>
+      {% for doc in latest_docs %}
+        {% include "message-preview.html" %}
+      {% endfor %}
+    </ul>
+    <hr class="space"/>
+    <a class="right" href="{% url search %}?o=modified+desc">
+      {% trans "Ver mais colaborações..." %}
+    </a>
+  </div>
+
+  <div class="span-11 last">
+    <h3>{% trans "Distribuição das Colaborações:" %}</h3>
+    <div id="collabs"></div>
+  </div>
+
   <hr class="space"/>
-  <div class="span-13">
-    <a class="right append-1" href="{% url super_archives.views.list_messages %}">
-      {% trans "Ver mais mensagens..." %}
+  <hr/>
+
+  <div class="span-12 colborder">
+    <h3>{% trans "Discussões Mais Relevantes:" %}</h3>
+    <ul>
+      {% for thread in hotest_threads %}
+        {% include "message-preview.html" with doc=thread.latest_message %}
+      {% endfor %}
+    </ul>
+    <hr class="space"/>
+    <a class="right" href="{% url thread_list %}?order=hotest">
+      {% trans "Ver mais discussões relevantes..." %}
     </a>
   </div>
+
   <div class="span-11 last">
-    <a class="right" href="{% url super_archives.views.list_messages %}?order=hotest">
-      {% trans "Ver mais mensagens..." %}
+    <h3>{% trans "Últimas Discussões:" %}</h3>
+    <ul>
+      {% for thread in latest_threads %}
+        {% include "message-preview.html" with doc=thread.latest_message %}
+      {% endfor %}
+    </ul>
+    <hr class="space"/>
+    <a class="right" href="{% url thread_list %}">
+      {% trans "Ver mais discussões..." %}
     </a>
   </div>
+
 {% endblock %}
@@ -10,8 +10,8 @@
   {% endif %}
  
   <div id="login-form-wrapper">
-    <form class="span-24" method="post" action="{% url django.contrib.auth.views.login %}">
-    {% csrf_token %}
+    <form class="span-24" method="POST" action="{% url login %}">
+      {% csrf_token %}
  
       <fieldset class="span-10 box">
         <legend>Login</legend>
@@ -27,8 +27,9 @@
         </p>
  
         <input type="submit" value="Login" />
-        {% url colab.views.user_profile_empty as user_profile_url %}
-        <input type="hidden" name="next" value="{% firstof next_page user_profile_url %}" />
+        <a class="right" href="{% url request_reset_password %}">Esqueci minha senha</a>
+        
+        <input type="hidden" name="next" value="{% firstof next '/' %}" />
       </fieldset>
  
       <fieldset class="span-12 box last">
@@ -39,8 +40,8 @@
         comunidade Interlegis clique no link abaixo e comece a colaborar!</p>
  
         <a class="span-11 center large" 
-           href="{% url colab.views.signup %}">Cadastre-se</a>
+           href="{% url signup %}">Cadastre-se</a>
       </fieldset>
     </form>
   </div>
-{% endblock %}
 \ No newline at end of file
+{% endblock %}
@@ -0,0 +1,45 @@
+{% load i18n %}
+
+{% if type_count %}
+    <!--Load the AJAX API-->
+    <script type="text/javascript" src="https://www.google.com/jsapi"></script>
+    <script type="text/javascript">
+    
+      // Load the Visualization API and the piechart package.
+      google.load('visualization', '1.0', {'packages':['corechart']});
+      
+      // Set a callback to run when the Google Visualization API is loaded.
+      google.setOnLoadCallback(drawChart);
+      
+      // Callback that creates and populates a data table, 
+      // instantiates the pie chart, passes in the data and
+      // draws it.
+      function drawChart() {
+
+      // Create the data table.
+      var data = new google.visualization.DataTable();
+      data.addColumn('string', 'Topping');
+      data.addColumn('number', 'Slices');
+      data.addRows([
+        ['{% trans "Emails"%}', {% firstof type_count.thread '0' %}],
+        ['{% trans "Tiquetes"%}', {% firstof type_count.ticket '0' %}],
+        ['{% trans "Wiki"%}', {% firstof type_count.wiki '0' %}],
+        ['{% trans "Código"%}', {% firstof type_count.changeset '0' %}],
+      ]);
+
+      // Set chart options
+      var options = {'width': {{ chart_width }},
+                     'height': {{ chart_height }},
+                     'legend': 'bottom',
+                     chartArea: {
+                        top:10,
+                        height:"85%",
+                      },
+      };
+
+      // Instantiate and draw our chart, passing in some options.
+      var chart = new google.visualization.PieChart(document.getElementById('{{ chart_div }}'));
+      chart.draw(data, options);
+    }
+    </script>
+{% endif %}
@@ -0,0 +1,78 @@
+{% extends "base.html" %}
+{% load i18n %}
+{% load append_to_get %}
+{% block main-content %}
+  <h2 class="span-6">{% trans "Busca" %}</h2>
+  <span class="right quiet">
+    {{ docs.numFound }} {% trans "documentos encontrados em" %} 
+    {{ docs.QTime|floatformat:3 }} {% trans "segundos" %}
+  </span>
+
+  <hr/>
+
+  <div class="span-5 border filters">
+    <h3>Filtros</h3>
+
+    <h4>{% trans "Tipos" %}</h4>
+
+    <ul class="legend">
+      <li {% ifequal type "wiki" %} class="selected" title="{% trans "Retirar Filtro" %}" {% endifequal %}>
+        <span>
+          <img src="{{ STATIC_URL }}img/wiki.png">
+        </span>
+        <a href="{% ifnotequal type "wiki" %} {% append_to_get type='wiki' %} {% else %} {% append_to_get type="" %} {% endifnotequal %}">{% trans "Wiki" %}</a>
+      </li>
+      <li {% ifequal type "thread" %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+        <span>
+          <img src="{{ STATIC_URL }}img/thread.png"> 
+        </span>
+        <a href="{% ifnotequal type "thread" %} {% append_to_get type='thread' %} {% else %} {% append_to_get type="" %} {% endifnotequal %}">{% trans "Discussão" %}</a>
+      </li>
+      <li {% ifequal type "ticket" %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+        <span>
+          <img src="{{ STATIC_URL }}img/ticket.png">
+        </span>
+        <a href="{% ifnotequal type "ticket" %} {% append_to_get type='ticket' %} {% else %} {% append_to_get type="" %} {% endifnotequal %}">{% trans "Tiquete" %}</a>
+      </li>
+      <li {% ifequal type "changeset" %} class="selected" title="{% trans "Retirar filtro" %}" {% endifequal %}>
+        <span>
+          <img src="{{ STATIC_URL }}img/changeset.png"> 
+        </span>
+        <a href="{% ifnotequal type "changeset" %} {% append_to_get type='changeset' %} {% else %} {% append_to_get type="" %} {% endifnotequal %}">{% trans "Changeset" %}</a>
+      </li>
+    </ul>
+  </div>
+  
+  <div class="span-17 prepend-1 last">
+    <ul>
+    {% for doc in docs  %}
+      {% include "message-preview.html" %}
+    {% empty %}
+      <li class="center">
+        {% trans "Sem resultados para a busca." %}
+      </li>
+    {% endfor %}
+    </ul>
+
+    <hr class="space"/>
+  
+    {% if docs.numFound %}
+    <div class="pagination center">
+      <span>
+        {% if docs.has_previous %}
+          <a href="{% append_to_get p=docs.previous_page_number %}">{% trans "Anterior" %}</a>
+        {% endif %}
+
+        <span>
+          {% trans "Página" %} {{ docs.page_num }} {% trans "de" %} {{ docs.num_of_pages }}
+        </span>
+
+        {% if docs.has_next %}
+          <a href="{% append_to_get p=docs.next_page_number %}">{% trans "Próxima" %}</a>
+        {% endif %}
+      </span>
+    </div>
+    {% endif %}
+  </div>
+
+{% endblock %}
 {% extends "base.html" %}
 {% load form_field %}
+{% load i18n %}
 {% block main-content %}
  
 <h2>Cadastre-se</h2>
@@ -8,6 +9,17 @@
 <div class="alert"><b>Por favor, corrija os erros abaixo e tente novamente.</b></div>
 {% endif %}
  
+<div class="avatar-placeholder box">
+  <label class="avatar-placeholder">Avatar</label >
+  <div class="avatar-image avatar">
+    <img  src="{{ STATIC_URL }}img/user.png" alt="user"/>
+  </div>
+  <p>
+    {% trans "Adicione um avatar à sua conta utilizando" %} <a href="http://pt.gravatar.com/" target="_blank">Gravatar</a>.
+  </p>
+</div>
+
+
 <p class="required span-24 last">
   <label>Campos Obrigatórios</label>  
 </p>
@@ -47,4 +59,4 @@
     <input class="right" type="submit" value="Cadastrar"/>
   </div>
 </form>
-{% endblock %}
 \ No newline at end of file
+{% endblock %}
 {% extends "base.html" %}
+{% load i18n %}
 {% load form_field %}
  
 {% block head_js %}
-    <!--Load the AJAX API-->
-    <script type="text/javascript" src="https://www.google.com/jsapi"></script>
-    <script type="text/javascript">
-    
-      // Load the Visualization API and the piechart package.
-      google.load('visualization', '1.0', {'packages':['corechart']});
-      
-      // Set a callback to run when the Google Visualization API is loaded.
-      google.setOnLoadCallback(drawChart);
-      
-      // Callback that creates and populates a data table, 
-      // instantiates the pie chart, passes in the data and
-      // draws it.
-      function drawChart() {
-
-      // Create the data table.
-      var data = new google.visualization.DataTable();
-      data.addColumn('string', 'Topping');
-      data.addColumn('number', 'Slices');
-      data.addRows([
-        ['Escolhida pelo autor', 21],
-        ['Bem avaliadas', 50],
-        ['Outras', 230],
-      ]);
-
-      // Set chart options
-      var options = {'width': 350,
-                     'height': 200,
-                     'legend': 'bottom',
-                     chartArea: {
-                        top:10, 
-                        width:"90%", 
-                        height:"75%"},
-      };
-
-      // Instantiate and draw our chart, passing in some options.
-      var chart = new google.visualization.PieChart(document.getElementById('chart_div'));
-      chart.draw(data, options);
-    }
-    </script>
+  {% include "pizza-chart.html" with chart_div="collabs" chart_width=390 chart_height=230 %}
 {% endblock %}
  
-
 {% block main-content %}
   {% if not user_profile %}
     <span class="notice span-24">
       <b>Usuário não cadastrado.</b> Você é dono deste perfil? 
-      <a href="{% url colab.views.signup %}">Clique aqui
+      <a href="{% url signup %}">Clique aqui
       e cadastre-se.</a>
     </span>
+
+  {% else %}  
+    
+    {% ifequal request.user.username user_profile.user.username %}
+      <span class="success span-24">
+        Ei, olha você aqui! Quer 
+        <a href="{% url user_profile_update request.user %}">editar seu perfil</a>?
+      </span>
+    {% endifequal %}
+    
   {% endif %}
  
   <div id="user-profile">
@@ -63,13 +34,18 @@
     </div>
  
     <div class="span-20 last">
-      <div class="span-11">
-        <form action='.' method='post'>
+      <div class="span-10">
+        <form action="{% url user_profile_update request.user %}" method='post'>
           {% csrf_token %}
  
           <h3>Informações Pessoais</h3>
           <ul id="user-info">
-            <li>{{ form.username.label_tag }}: {{ user_profile.user.username }}</li>
+            <li>
+              {{ form.username.label_tag }}: {{ user_profile.user.username }}
+              {% ifequal request.user.username user_profile.user.username %}
+                (<a href="{% url change_password %}">{% trans "alterar senha" %}</a>)
+              {% endifequal %}
+            </li>
             <li>
               {% render_form_field form.institution user_profile.institution %}
             </li>
@@ -105,21 +81,37 @@
         </form>
       </div>
  
-      <div class="span-9 last">
-        <h3 class="center">Avaliação das Menssagens</h3>
-        <div id="chart_div"></div>
+      {% if type_count %}
+      <div class="span-10 last">
+        <h3 class="center">{% trans "Colaborações por área" %}</h3>
+        <div id="collabs"></div>
       </div>
+      {% endif %}
     </div>
  
     <hr class="space" />
  
-    <h3>Últimas mensagens enviadas</h3>
-    <ul>
-    {% for email in emails %}
-      {% include "message-preview.html" %}
-    {% empty %}
-      <li>Não existem mensagens enviadas por este usuário até o momento.</li>
-    {% endfor %} 
-    </ul>
+    <div class="span-13">
+      <h3>{% trans "Últimas mensagens enviadas" %} </h3>
+      <ul class="colborder">
+      {% for doc in emails %}
+        {% include "message-preview.html" %}
+      {% empty %}
+        <li>Não existem mensagens enviadas por este usuário até o momento.</li>
+      {% endfor %} 
+      </ul>
+    </div>
+    
+    <div class="span-11 last">
+      <h3>{% trans "Participações na comunidade:" %}</h3>
+      <ul>
+      {% for doc in docs %}
+        {% include "message-preview.html" %}
+      {% empty %}
+        <li>Sem colaborações deste usuário até o momento.</li>
+      {% endfor %}
+      </ul>
+    </div>
+    
   </div>
-{% endblock %}
 \ No newline at end of file
+{% endblock %}
@@ -5,31 +5,48 @@ from django.contrib import admin
 admin.autodiscover()
  
 urlpatterns = patterns('',
-    url(r'^$', 'colab.views.home', name='home'),
+    url(r'^$', 'colab.views.other.home', name='home'),
  
     url(r'^archives/', include('colab.super_archives.urls')),
  
     url(r'^api/', include('colab.api.urls')),
  
+    url(r'^user/(?P<username>[\w@+.-]+)/?$',
+        'colab.views.userprofile.by_username', name='user_profile'),
+    
+    url(r'^user/$', 'colab.views.userprofile.by_request_user', 
+        name='user_profile_by_request_user'),
+    
     url(r'^user/hash/(?P<emailhash>[\w]+)$',
-        'colab.views.user_profile_emailhash'),
-
-    url(r'^user/(?P<username>[\w]+)/?$', 'colab.views.user_profile_username'),
+        'colab.views.userprofile.by_emailhash'),
  
     url(r'^user/(?P<username>[\w]+)/edit/?$', 
-        'colab.views.user_profile_edit'),
+        'colab.views.userprofile.update', name='user_profile_update'),
+        
+    url(r'^search/$', 'colab.views.other.search', name='search'),
  
-    url(r'^user/$',
-        'colab.views.user_profile_empty'),
+    url(r'^account/$', 'colab.views.signup.signup', name='signup'), 
  
-    url(r'^signup/$', 'colab.views.signup'), 
+    url(r'^account/changepassword/$', 'colab.views.signup.change_password',
+        name='change_password'),
  
-    url(r'^login/$', 'django.contrib.auth.views.login', 
-        {'template_name': 'login.html'}),
+    url(r'^account/resetpassword/$', 
+        'colab.views.signup.request_reset_password', 
+        name='request_reset_password'),
+        
+    url(r'^account/reset_password/(?P<hash>[\w]{32})/$',
+        'colab.views.signup.reset_password', name='reset_password'),
  
-    url(r'^logout/$', 'django.contrib.auth.views.logout', 
-        {'next_page': '/'}),
+    url(r'^signup/verify/(?P<hash>[\w]{32})/$', 
+        'colab.views.signup.verify_email', name='email_verification'), 
+
+    url(r'^account/login/$', 'django.contrib.auth.views.login', 
+        {'template_name': 'login.html'}, name='login'),
+
+    url(r'^account/logout/$', 'django.contrib.auth.views.logout', 
+        {'next_page': '/'}, name='logout'),
  
     # Uncomment the next line to enable the admin:
-    #url(r'^admin/', include(admin.site.urls)),
+    url(r'^colab/admin/', include(admin.site.urls)),
 )
+
@@ -1,159 +0,0 @@
-
-from django.template import RequestContext
-from django.contrib.auth.models import User
-from django.forms.models import model_to_dict
-from django.contrib.auth.decorators import login_required
-from django.shortcuts import render_to_response, get_object_or_404, redirect
-
-from super_archives import queries
-from super_archives.forms import UserCreationForm, UserUpdateForm
-from super_archives.models import Message, Thread, UserProfile, EmailAddress
-
-
-def home(request):
-    """Index page view"""
-    
-    latest_threads = queries.get_latest_threads()
-    hotest_threads = queries.get_voted_threads()
-    
-    template_data = {
-        'hotest_threads': hotest_threads[:6],
-        'latest_threads': latest_threads[:6],
-    }
-    return render_to_response('home.html', template_data, 
-                              context_instance=RequestContext(request))
-                              
-
-def signup(request):
-
-    # If the request method is GET just return the form
-    if request.method == 'GET':
-        form = UserCreationForm()
-        return render_to_response('signup-form.html', {'form': form}, 
-                                  RequestContext(request))
-
-    # If the request method is POST try to store data
-    form = UserCreationForm(request.POST)
-    
-    # If there is validation errors give the form back to the user
-    if not form.is_valid():
-        return render_to_response('signup-form.html', {'form': form}, 
-                                  RequestContext(request))
-
-    user = User(
-        username=form.cleaned_data.get('username'),
-        email=form.cleaned_data.get('email'),
-        first_name=form.cleaned_data.get('first_name'),
-        last_name=form.cleaned_data.get('last_name'),
-    )
-    user.set_password(form.cleaned_data.get('password'))
-    user.save()
-    
-    profile = UserProfile(
-        user=user,
-        institution=form.cleaned_data.get('institution'),
-        role=form.cleaned_data.get('role'),
-        twitter=form.cleaned_data.get('twitter'),
-        facebook=form.cleaned_data.get('facebook'),
-        google_talk=form.cleaned_data.get('google_talk'),
-        webpage=form.cleaned_data.get('webpage'),
-    )
-    profile.save()
-    
-    # Check if the user's email have been used previously 
-    #   in the mainling lists to link the user to old messages  
-    email_addr, created = EmailAddress.objects.get_or_create(address=user.email)
-    if created:
-        email_addr.real_name = user.get_full_name()
-
-    email_addr.user = user
-    email_addr.save()
-
-    return redirect('colab.views.user_profile_username', user.username)
-
-
-def user_profile(request, user, email_address=None, editable=False, form=None):
-    
-    if form is None:
-        form = UserCreationForm()
-    
-    if user:
-        email_addresses = user.emails.all()
-    else:
-        email_addresses = [email_address]
-        
-    if not email_address:
-        email_address = email_addresses[0]
-
-    email_addresses_ids = tuple([str(addr.id) for addr in email_addresses])
-
-    query = """
-        SELECT
-            * 
-        FROM
-            super_archives_message
-        WHERE 
-            from_address_id IN (%(ids)s)
-        GROUP BY
-            thread_id
-        ORDER BY
-            received_time DESC
-        LIMIT 10;
-    """ % {'ids': ','.join(email_addresses_ids)}
-
-    emails = Message.objects.raw(query)
-    n_sent = Message.objects.filter(from_address__in=email_addresses).count()    
-
-    if user:
-        profile = user.profile
-    else:
-        profile = None
-
-    template_data = {
-        'user_profile': profile,
-        'email_address': email_address,
-        'emails': emails or [],
-        'form': form,
-        'editable': editable,
-    }
-    return render_to_response('user-profile.html', template_data, 
-                              RequestContext(request))
-
-
-@login_required
-def user_profile_empty(request):
-    return user_profile(request, request.user)
-
-
-def user_profile_username(request, username):
-    user = get_object_or_404(User, username=username)
-    return user_profile(request, user)
-
-
-def user_profile_emailhash(request, emailhash):
-    email_addr = get_object_or_404(EmailAddress, md5=emailhash)
-    return user_profile(request, email_addr.user, email_addr)
-
-
-@login_required
-def user_profile_edit(request, username):
-    profile = get_object_or_404(UserProfile, user__username=username)
-    form = UserUpdateForm(initial=model_to_dict(profile))
-    
-    if request.method == "GET":
-        return user_profile(request, profile.user, editable=True, form=form)
-
-    form = UserUpdateForm(request.POST)
-    if not form.is_valid():
-        return user_profile(request, profile.user, editable=True, form=form)
-
-    profile.institution = form.cleaned_data.get('institution')
-    profile.role = form.cleaned_data.get('role')
-    profile.twitter = form.cleaned_data.get('twitter')
-    profile.facebook = form.cleaned_data.get('facebook')
-    profile.google_talk = form.cleaned_data.get('google_talk')
-    profile.webpage = form.cleaned_data.get('webpage')
-    profile.save()
-    
-    return redirect('colab.views.user_profile_username', profile.user.username)
-    
 \ No newline at end of file
@@ -0,0 +1,2 @@
+#!/usr/bin/env python
+# encoding: utf-8
@@ -0,0 +1,61 @@
+#!/usr/bin/env python
+# encoding: utf-8
+"""
+other.py
+
+Created by Sergio Campos on 2012-01-10.
+"""
+
+from django.template import RequestContext
+from django.shortcuts import render_to_response
+from django.utils.translation import ugettext as _
+
+from colab import solrutils
+from colab.super_archives import queries
+
+
+def home(request):
+    """Index page view"""
+
+    latest_threads = queries.get_latest_threads()
+    hotest_threads = queries.get_hotest_threads()
+
+    template_data = {
+        'hotest_threads': hotest_threads[:6],
+        'latest_threads': latest_threads[:6],
+        'type_count': solrutils.count_types(sample=1000),
+        'latest_docs': solrutils.get_latest_collaborations(6),
+    }
+    return render_to_response('home.html', template_data,
+                              context_instance=RequestContext(request))
+
+
+def search(request):
+    if request.method == 'GET':
+        query = request.GET.get('q')
+        sort = request.GET.get('o')
+        type_ = request.GET.get('type')
+        page_number = int(request.GET.get('p', 1))
+        results_per_page = int(request.GET.get('per_page', 16))
+
+    filters = {
+        'Type': type_,
+    }
+
+    query = solrutils.build_query(query, filters)
+
+    # Query Solr for results
+    solr_dict_resp = solrutils.select(query, results_per_page,
+                                      page_number, sort)
+
+    docs = solrutils.SolrPaginator(solr_dict_resp, page_number)
+
+    template_data = {
+        'docs': docs,
+        'anonymous': _(u'anônimo'),
+        'q': query,
+        'type': type_,
+    }
+
+    return render_to_response('search.html', template_data,
+                              RequestContext(request))
@@ -0,0 +1,218 @@
+#!/usr/bin/env python
+# encoding: utf-8
+"""
+signup.py
+
+Created by Sergio Campos on 2012-01-10.
+"""
+
+import uuid
+from colab import signup as signup_
+
+from django.template import RequestContext
+from django.contrib.auth.models import User
+from django.utils.translation import ugettext as _
+from django.contrib.auth.decorators import login_required
+from django.contrib.auth.forms import SetPasswordForm, PasswordChangeForm
+from django.shortcuts import render_to_response, redirect, get_object_or_404
+
+from colab.super_archives.forms import UserCreationForm
+from colab.super_archives.models import UserProfile, EmailAddress
+
+
+def signup(request):
+
+    # If the request method is GET just return the form
+    if request.method == 'GET':
+        form = UserCreationForm()
+        return render_to_response('signup-form.html', {'form': form}, 
+                                  RequestContext(request))
+
+    # If the request method is POST try to store data
+    form = UserCreationForm(request.POST)
+    
+    # If there is validation errors give the form back to the user
+    if not form.is_valid():
+        return render_to_response('signup-form.html', {'form': form}, 
+                                  RequestContext(request))
+
+    user = User(
+        username=form.cleaned_data.get('username'),
+        email=form.cleaned_data.get('email'),
+        first_name=form.cleaned_data.get('first_name'),
+        last_name=form.cleaned_data.get('last_name'),
+        is_active=False,
+    )
+    user.set_password(form.cleaned_data.get('password1'))
+    user.save()
+    
+    profile = UserProfile(
+        user=user,
+        institution=form.cleaned_data.get('institution'),
+        role=form.cleaned_data.get('role'),
+        twitter=form.cleaned_data.get('twitter'),
+        facebook=form.cleaned_data.get('facebook'),
+        google_talk=form.cleaned_data.get('google_talk'),
+        webpage=form.cleaned_data.get('webpage'),
+        verification_hash=uuid.uuid4().get_hex(),
+    )
+    profile.save()
+    
+    signup_.send_verification_email(request, user)
+
+    # Check if the user's email have been used previously 
+    #   in the mainling lists to link the user to old messages  
+    email_addr, created = EmailAddress.objects.get_or_create(address=user.email)
+    if created:
+        email_addr.real_name = user.get_full_name()
+
+    email_addr.user = user
+    email_addr.save()
+    
+    template_data = {
+        'msg': _((u'Cadastro efetuado com sucesso. Por favor acesse seu '
+                u'endereço de email para validá-lo.')),
+        'msg_css_class': 'success', 
+    }
+    
+    return render_to_response('account_message.html', template_data,
+                              RequestContext(request))
+
+
+def verify_email(request, hash):
+    """Verify hash and activate user's account"""
+    
+    profile = get_object_or_404(UserProfile, verification_hash=hash)
+    
+    profile.verification_hash = 'verified'
+    profile.save()
+    
+    profile.user.is_active = True
+    profile.user.save()
+    
+    template_data = {
+        'msg': _(u'Endereço de e-mail validado corretamente.'),
+        'msg_css_class': 'success', 
+    }
+    
+    return render_to_response('account_message.html', template_data,
+                              RequestContext(request))
+    
+
+def request_reset_password(request):
+    """Request a password reset.
+    
+    In case request method is GET it will display the password reset
+    form. Otherwise we'll look for a username in the POST request to
+    have its password reset. This user will receive a link where he
+    will be allowed to change his password.
+    
+    """
+    
+    if request.method == 'GET':
+        return render_to_response('account_request_reset_password.html', {},
+                                  RequestContext(request))
+    
+    username = request.POST.get('username')
+
+    try:
+        user = User.objects.get(username=username)
+    except User.DoesNotExist:
+        user = None
+    
+    if user and user.is_active:
+        profile = user.profile
+        profile.verification_hash = uuid.uuid4().get_hex()
+        profile.save()
+                
+        signup_.send_reset_password_email(request, user)
+    
+    msg = _((u'Para sua segurança, dentro de alguns instantes você '
+           u'receberá um email solicitando a confirmação do pedido '
+           u'de troca de senha. Por favor aguarde.'))
+    
+    template_data = {
+        'msg': msg,
+        'msg_css_class': 'info', 
+    }
+    
+    return render_to_response('account_message.html', template_data,
+                              RequestContext(request))
+
+
+def reset_password(request, hash):
+    """Perform a password change.
+    
+    If the request method is set to GET and the hash matches a form 
+    will be displayed to the user allowing the password change.
+    If the request method is POST the user password will be changed
+    to the newly set data.
+    
+    """
+    
+    profile = get_object_or_404(UserProfile, verification_hash=hash)
+    user = profile.user
+    
+    form = SetPasswordForm(profile.user)
+    
+    template_data = {
+        'form': form,
+        'hash': hash,
+    }
+    
+    if request.method == 'GET':
+        return render_to_response('account_change_password.html', 
+                                  template_data, RequestContext(request))
+
+
+    form = SetPasswordForm(user, request.POST)
+    template_data.update({'form': form})
+    
+    if not form.is_valid():
+        return render_to_response('account_change_password.html', 
+                                  template_data, RequestContext(request))
+
+    profile.verification_hash = 'verified'
+    profile.save()
+
+    user.set_password(form.cleaned_data.get('new_password1'))
+    user.save()
+
+    template_data.update({
+        'msg': _(u'Senha alterada com sucesso!'),
+        'msg_css_class': 'success',
+    })
+    return render_to_response('account_message.html', template_data,
+                              RequestContext(request))
+
+@login_required
+def change_password(request):
+    """Change a password for an authenticated user."""
+    
+    form = PasswordChangeForm(request.user)
+    
+    template_data = {
+        'form': form
+    }
+    
+    if request.method == 'GET':
+        return render_to_response('account_change_password.html',
+                                  template_data, RequestContext(request))
+    
+    form = PasswordChangeForm(request.user, request.POST)
+    template_data.update({'form': form})
+    
+    if not form.is_valid():
+        return render_to_response('account_change_password.html', 
+                                  template_data, RequestContext(request))
+                                  
+    request.user.set_password(form.cleaned_data.get('new_password1'))
+    request.user.save()
+    
+    template_data.update({
+        'msg': _(u'Senha alterada com sucesso!'),
+        'msg_css_class': 'success',
+    })
+    return render_to_response('account_message.html', template_data,
+                              RequestContext(request))    
+    
@@ -0,0 +1,116 @@
+#!/usr/bin/env python
+# encoding: utf-8
+"""
+userprofile.py
+
+Created by Sergio Campos on 2012-01-10.
+"""
+
+from django.template import RequestContext
+from django.contrib.auth.models import User
+from django.forms.models import model_to_dict
+from django.contrib.auth.decorators import login_required
+from django.shortcuts import render_to_response, get_object_or_404, redirect
+
+from colab import solrutils
+from colab.super_archives.forms import UserCreationForm, UserUpdateForm
+from colab.super_archives.models import Message, UserProfile, EmailAddress
+                                        
+
+def read(request, user, email_address=None, editable=False, form=None):
+    
+    if form is None:
+        form = UserCreationForm()
+    
+    if user:
+        email_addresses = user.emails.all()
+        profile = user.profile
+        last_modified_docs = solrutils.get_latest_collaborations(
+            username=user.username
+        )
+        
+        type_count = solrutils.count_types(
+            filters={'collaborator': user.username}
+        )
+        
+    else:
+        email_addresses = [email_address]
+        profile = None
+        last_modified_docs = []
+        type_count = {}
+        
+    if not email_address and email_addresses:
+        email_address = email_addresses[0]
+
+    email_addresses_ids = tuple([str(addr.id) for addr in email_addresses])
+    
+    query = """
+    SELECT 
+        * 
+    FROM
+        super_archives_message JOIN (
+            SELECT id
+            FROM super_archives_message
+            WHERE from_address_id IN (%(ids)s)
+            GROUP BY thread_id, id
+        ) AS subquery 
+        ON subquery.id = super_archives_message.id
+    ORDER BY 
+        received_time DESC
+    LIMIT 10;
+    
+    """ % {'ids': ','.join(email_addresses_ids)}
+
+    emails = Message.objects.raw(query)
+    #n_sent = Message.objects.filter(from_address__in=email_addresses).count()
+
+    template_data = {
+        'user_profile': profile,
+        'email_address': email_address,
+        'emails': emails or [],
+        'form': form,
+        'editable': editable,
+        'type_count': type_count,
+        'docs': last_modified_docs,
+    }
+    
+    return render_to_response('user-profile.html', template_data, 
+                              RequestContext(request))
+
+
+@login_required
+def by_request_user(request):
+    return read(request, request.user)
+
+
+def by_username(request, username):
+    user = get_object_or_404(User, username=username)
+    return read(request, user)
+
+
+def by_emailhash(request, emailhash):
+    email_addr = get_object_or_404(EmailAddress, md5=emailhash)
+    return read(request, email_addr.user, email_addr)
+
+
+@login_required
+def update(request, username):
+    profile = get_object_or_404(UserProfile, user__username=username)
+    form = UserUpdateForm(initial=model_to_dict(profile))
+    
+    if request.method == "GET":
+        return read(request, profile.user, editable=True, form=form)
+
+    form = UserUpdateForm(request.POST)
+    if not form.is_valid():
+        return read(request, profile.user, editable=True, form=form)
+
+    profile.institution = form.cleaned_data.get('institution')
+    profile.role = form.cleaned_data.get('role')
+    profile.twitter = form.cleaned_data.get('twitter')
+    profile.facebook = form.cleaned_data.get('facebook')
+    profile.google_talk = form.cleaned_data.get('google_talk')
+    profile.webpage = form.cleaned_data.get('webpage')
+    profile.save()
+    
+    return redirect('user_profile', profile.user.username)
@@ -0,0 +1,64 @@
+<VirtualHost *:80>
+    ServerName colab01a.interlegis.leg.br
+    ServerAlias colab.interlegis.leg.br
+    ServerAlias colab.interlegis.gov.br
+
+    <Proxy *>
+        Order deny,allow
+        Allow from all
+    </Proxy>
+
+	RewriteEngine On
+	RewriteRule ^/admin(.*) http://colab-backend.interlegis.leg.br/admin$1 [P]
+	RewriteRule ^/wiki(.*) http://colab-backend.interlegis.leg.br/wiki$1 [P]
+	RewriteRule ^/changeset(.*) http://colab-backend.interlegis.leg.br/changeset$1 [P]
+	RewriteRule ^/newticket(.*) http://colab-backend.interlegis.leg.br/newticket$1 [P]
+	RewriteRule ^/ticket(.*) http://colab-backend.interlegis.leg.br/ticket$1 [P]
+	RewriteRule ^/chrome(.*) http://colab-backend.interlegis.leg.br/chrome$1 [P]
+	RewriteRule ^/timeline(.*) http://colab-backend.interlegis.leg.br/timeline$1 [P]
+	RewriteRule ^/roadmap(.*) http://colab-backend.interlegis.leg.br/roadmap$1 [P]
+	RewriteRule ^/browser(.*) http://colab-backend.interlegis.leg.br/browser$1 [P]
+	RewriteRule ^/report(.*) http://colab-backend.interlegis.leg.br/report$1 [P]
+	RewriteRule ^/tags(.*) http://colab-backend.interlegis.leg.br/tags$1 [P]
+	RewriteRule ^/query(.*) http://colab-backend.interlegis.leg.br/query$1 [P]
+	RewriteRule ^/about(.*) http://colab-backend.interlegis.leg.br/about$1 [P]
+	RewriteRule ^/prefs(.*) http://colab-backend.interlegis.leg.br/prefs$1 [P]
+	RewriteRule ^/login(.*) http://colab-backend.interlegis.leg.br/login$1 [P]
+	RewriteRule ^/logout(.*) http://colab-backend.interlegis.leg.br/logout$1 [P]
+	RewriteRule ^/attachment(.*) http://colab-backend.interlegis.leg.br/attachment$1 [P]
+	RewriteRule ^/raw-attachment(.*) http://colab-backend.interlegis.leg.br/raw-attachment$1 [P]
+
+    WSGIDaemonProcess colab user=www-data group=www-data threads=25
+    WSGIProcessGroup colab
+    
+    Alias /static/admin/ /usr/local/django/colab/lib/python2.6/site-packages/django/contrib/admin/media/
+    <Directory /usr/local/django/colab/lib/python2.6/site-packages/django/contrib/admin/media/>
+        Options -Indexes
+        Order deny,allow
+        Allow from all
+        AllowOverride None
+    </Directory>
+
+    Alias /static/ /usr/local/django/colab/static/
+    <Directory /usr/local/django/colab/static/>
+        Options -Indexes
+        Order deny,allow
+        Allow from all
+        AllowOverride None
+    </Directory>
+
+    WSGIScriptAlias / /usr/local/django/colab/wsgi/colab.wsgi
+    <Directory /usr/local/django/colab/wsgi/>
+        Order deny,allow
+        Allow from all
+        AllowOverride None
+    </Directory>
+
+    ErrorLog /var/log/apache2/error.log
+
+    # Possible values include: debug, info, notice, warn, error, crit,
+    # alert, emerg.
+    LogLevel warn
+
+    CustomLog /var/log/apache2/access.log combined
+</VirtualHost>
@@ -0,0 +1,12 @@
+import site
+import os
+
+cwd_path = os.path.abspath(os.path.dirname(__file__))
+virtualenv_path = os.path.join(cwd_path, '../lib/python2.6/site-packages')
+site.addsitedir(virtualenv_path)
+
+from django.core.handlers.wsgi import WSGIHandler
+os.environ['DJANGO_SETTINGS_MODULE'] = 'colab.settings'
+application = WSGIHandler()
+
+
@@ -0,0 +1 @@
+archives -fstype=fuse,rw,nodev,users,noatime :sshfs\#root@listas.interlegis.gov.br\:/var/lib/mailman/archives/private
@@ -0,0 +1,2 @@
+PATH="/usr/local/django/colab/bin"
+* * * * * root DJANGO_SETTINGS_MODULE="colab.settings" django-admin.py import_emails --archives_path=/usr/local/django/colab/mnt/archives/ --exclude-list=saberes-divulgacao --exclude-list=pml --exclude-list=mailman --exclude-list=lexml-anuncios > /tmp/colab-import-emails-last.log
@@ -0,0 +1,3 @@
+PATH="/usr/bin"
+*/1 * * * * root wget "http://solr.interlegis.leg.br:8080/solr/dataimport?wt=json&indent=true&command=delta-import" -q -O - >> /tmp/solr-delta-import.log
+4 6 * * * root wget "http://solr.interlegis.leg.br:8080/solr/dataimport?wt=json&indent=true&command=full-import" -q -O - >> /tmp/solr-full-import.log
 Django==1.3.1
 South==0.7.3
-django-registration==0.7
 django-piston==0.2.3
-pytz
+pytz==2011n
 chardet==1.0.1
+python-dateutil==1.5
+django-cliauth==0.9
@@ -0,0 +1,49 @@
+#!/usr/bin/env python
+
+from setuptools import setup, find_packages
+from setupext import Data_Files, install_Data_Files
+
+setup(name='colab',
+      version='0.1-dev',
+      author='Sergio Oliveira',
+      author_email='seocam@seocam.com',
+      url='https://bitbucket.org/seocam/atu-colab',
+      packages=find_packages(),
+      package_data={'colab': ['templates/*.html', 
+                              'super_archives/templates/*.html',
+                              'super_archives/fixtures/initial_data.json']},
+      data_files=[
+          Data_Files(base_dir='install_data',
+                     copy_to='static',
+                     template=['recursive-include colab/static *'],
+                     preserve_path=1,
+                     strip_dirs=2),
+          Data_Files(base_dir='install_data',
+                     copy_to='wsgi',
+                     template=['include etc/apache2/wsgi/colab.wsgi'],
+                     preserve_path=0), 
+          Data_Files(base_dir='install_data',
+                     copy_to='apache-site',
+                     template=['include etc/apache2/sites-available/colab'],
+                     preserve_path=0),
+          Data_Files(base_dir='install_data',
+                     copy_to='autofs',
+                     template=['include etc/autofs/listas'],
+                     preserve_path=0),
+          Data_Files(base_dir='install_data',
+                     copy_to='cron.d',
+                     template=['include etc/cron.d/*'],
+                     preserve_path=0),
+      ],
+      install_requires=(
+        'distribute',
+        'Django==1.3.1',
+        'South==0.7.3',
+        'django-piston==0.2.3',
+        'django-cliauth==0.9',
+        'pytz==2011n',
+        'chardet==1.0.1',
+        'python-dateutil==1.5',
+        'psycopg2==2.4.4'),
+      cmdclass = {"install_data": install_Data_Files},
+)
@@ -0,0 +1,15 @@
+"Package providing extensions to the standard distutils"
+
+from install_data import Data_Files, install_Data_Files
+
+from distutils.command.bdist_wininst import bdist_wininst
+
+# When building a windows installer, put some more text into
+# the long description
+class wininst_request_delete(bdist_wininst):
+    add_text = "\nIf you have installed earlier versions of this package, please remove them through 'Add/Remove Programs' before installing this release."
+
+    def get_inidata(self):
+        m = self.distribution.metadata
+        m.long_description = m.long_description + self.add_text
+        return bdist_wininst.get_inidata(self)
@@ -0,0 +1,197 @@
+# -*- coding: utf-8 -*-
+"""install_data.py
+
+Provides a more sophisticated facility to install data files
+than distutils' install_data does.
+You can specify your files as a template like in MANIFEST.in
+and you have more control over the copy process. 
+
+Copyright 2000 by Rene Liebscher, Germany.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+ 
+The above copyright notice and this permission notice shall be included
+in all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+"""
+
+# created 2000/08/01, Rene Liebscher <R.Liebscher@gmx.de>
+# modified 2000/12/18, Martin v. Löwis <loewis@informatik.hu-berlin.de>
+
+###########################################################################
+# import some modules we need 
+
+import os,sys,string
+from types import StringType,TupleType,ListType
+from distutils.util import change_root
+from distutils.filelist import FileList
+from distutils.command.install_data import install_data
+
+###########################################################################
+# a container class for our more sophisticated install mechanism
+
+class Data_Files:
+    """ container for list of data files.
+        supports alternate base_dirs e.g. 'install_lib','install_header',...
+        supports a directory where to copy files
+        supports templates as in MANIFEST.in
+        supports preserving of paths in filenames 
+            eg. foo/xyz is copied to base_dir/foo/xyz
+        supports stripping of leading dirs of source paths 
+            eg. foo/bar1/xyz, foo/bar2/abc can be copied to bar1/xyz, bar2/abc 
+    """
+
+    def __init__(self,base_dir=None,files=None,copy_to=None,template=None,preserve_path=0,strip_dirs=0):
+        self.base_dir = base_dir
+        self.files = files
+        self.copy_to = copy_to
+        if template is not None:
+            t = []
+            for item in template:
+                item = string.strip(item)
+                if not item:continue
+                t.append(item)
+            template = t
+        self.template = template
+        self.preserve_path = preserve_path
+        self.strip_dirs = strip_dirs
+        self.finalized = 0
+
+    def warn (self, msg):
+        sys.stderr.write ("warning: %s: %s\n" %
+                          ("install_data", msg))
+        
+    def debug_print (self, msg):
+        """Print 'msg' to stdout if the global DEBUG (taken from the
+        DISTUTILS_DEBUG environment variable) flag is true.
+        """
+        from distutils.core import DEBUG
+        if DEBUG:
+            print msg
+
+        
+    def finalize(self):
+        """ complete the files list by processing the given template """
+        if self.finalized: 
+            return
+        if self.files == None:
+            self.files = []
+        if self.template != None:
+            if type(self.template) == StringType:
+                self.template = string.split(self.template,";")
+            filelist = FileList(self.warn,self.debug_print)
+            for line in self.template:
+                filelist.process_template_line(string.strip(line))
+            filelist.sort()
+            filelist.remove_duplicates()    
+            self.files.extend(filelist.files)     
+        self.finalized = 1
+
+# end class Data_Files
+
+###########################################################################
+# a more sophisticated install routine than distutils install_data
+
+class install_Data_Files (install_data):
+    
+    def check_data(self,d):
+        """ check if data are in new format, if not create a suitable object.
+            returns finalized data object
+        """
+        if not isinstance(d, Data_Files):
+            self.warn(("old-style data files list found "
+                        "-- please convert to Data_Files instance"))
+            if type(d) is TupleType:
+                if len(d) != 2 or  not (type(d[1]) is ListType):
+                        raise DistutilsSetupError, \
+                          ("each element of 'data_files' option must be an "
+                            "Data File instance, a string or 2-tuple (string,[strings])")
+                d = Data_Files(copy_to=d[0],files=d[1])
+            else:
+                if not (type(d) is StringType):
+                        raise DistutilsSetupError, \
+                          ("each element of 'data_files' option must be an "
+                           "Data File instance, a string or 2-tuple (string,[strings])")
+                d = Data_Files(files=[d])
+        d.finalize()
+        return d
+     
+    def run(self):
+        self.outfiles = []
+        install_cmd = self.get_finalized_command('install')
+ 
+        for d in self.data_files:
+            d = self.check_data(d)
+ 
+            install_dir = self.install_dir
+            # alternative base dir given => overwrite install_dir
+            if d.base_dir != None:
+                install_dir = getattr(install_cmd,d.base_dir)
+     
+            # copy to an other directory 
+            if d.copy_to != None:
+                if not os.path.isabs(d.copy_to):
+                    # relatiev path to install_dir
+                    dir = os.path.join(install_dir, d.copy_to)
+                elif install_cmd.root:
+                    # absolute path and alternative root set 
+                    dir = change_root(self.root,d.copy_to)
+                else:
+                    # absolute path
+                    dir = d.copy_to                  
+            else:
+                # simply copy to install_dir
+                dir = install_dir
+                # warn if necceassary  
+                self.warn("setup script did not provide a directory to copy files to "
+                          " -- installing right in '%s'" % install_dir)
+ 
+            dir=os.path.normpath(dir)
+            # create path
+            self.mkpath(dir)
+ 
+            # copy all files    
+            for src in d.files:
+                if d.strip_dirs > 0:
+                    dst = string.join(string.split(os.path.normcase(src),os.sep)[d.strip_dirs:],os.sep)
+                else:
+                    dst = src
+                if d.preserve_path:
+                    # preserve path in filename
+                    self.mkpath(os.path.dirname(os.path.join(dir,dst)))
+                    out = self.copy_file(src, os.path.join(dir,dst))
+                else:
+                    out = self.copy_file(src, dir)
+                if type(out) is TupleType:
+                    out = out[0]
+                self.outfiles.append(out)
+ 
+        return self.outfiles
+ 
+    def get_inputs (self):
+        inputs = []
+        for d in self.data_files:
+            d = self.check_data(d)
+            inputs.append(d.files)
+        return inputs
+  
+    def get_outputs (self):
+         return self.outfiles
+
+
+###########################################################################
+      
@@ -0,0 +1,46 @@
+Installation instructions for Ubuntu 10.04
+-------------------------------------------
+
+* Install Java, tomcat, JDBC Postgres drivers (Ubuntu partner repositories must be enabled):
+sudo apt-get install sun-java6-bin tomcat6 libpg-java
+
+* Download Solr 3.3 and extract it:
+wget http://ftp.unicamp.br/pub/apache/lucene/solr/3.3.0/apache-solr-3.3.0.tgz
+tar xzf apache-solr-3.3.0.tgz 
+
+* Create the directory /var/local/lib/solr/ and give the right permissions:
+sudo mkdir -p /var/local/lib/solr/
+sudo chown tomcat6:tomcat6 /var/local/lib/solr/
+
+* Copy the solr home example to /usr/local/share/:
+sudo cp -R apache-solr-3.3.0/example/solr /usr/local/share/
+
+* Create a folder for libs in the solr home:
+sudo mkdir /usr/local/share/solr/lib/
+
+* Copy Solr libs to libs folder:
+sudo cp apache-solr-3.3.0/dist/*.jar /usr/local/share/solr/lib/
+
+* Copy Solr distribution to solr home:
+sudo cp apache-solr-3.3.0/dist/apache-solr-3.3.0.war /usr/local/share/solr/
+
+* Link the JDBC Postgres drivers into the Solr installation:
+sudo ln -s /usr/share/java/postgresql-jdbc3-8.4.jar /usr/local/share/solr/lib/
+
+* Link configurations to /etc
+sudo ln -s /usr/local/share/solr/conf/ /etc/solr
+
+* Copy the configuration files from this folder into /etc/solr/
+
+* Link the solr-tomcat.xml file in the Tomcat configuration:
+sudo ln -s /etc/solr/solr-tomcat.xml /etc/tomcat6/Catalina/localhost/solr.xml 
+
+* Check data-config.xml to make sure all information to connect to the databases are right
+
+* Create a dataimport.properties on /etc/solr and give write access to tomcat6:
+sudo touch /etc/solr/dataimport.properties
+sudo chown tomcat6:tomcat6 /etc/solr/dataimport.properties
+ 
+* Restart tomcat:
+sudo /etc/init.d/tomcat6 restart
+
@@ -0,0 +1,337 @@
+<dataConfig>
+  <dataSource name="trac"
+              type="JdbcDataSource" 
+              driver="org.postgresql.Driver"
+              url="jdbc:postgresql://bdinterlegis.interlegis.leg.br/trac_colab" 
+              user="colab" />
+  <dataSource name="colab"
+              type="JdbcDataSource" 
+              driver="org.postgresql.Driver"
+              url="jdbc:postgresql://bdinterlegis.interlegis.leg.br/colab" 
+              user="colab" />
+
+  <document>
+
+    <entity name="wiki"
+            dataSource="trac"
+            transformer="TemplateTransformer,DateFormatTransformer"
+            query="SELECT
+                     name,
+                     TIMESTAMP 'epoch' + (max(time)/1000000) * INTERVAL '1s' AS modified,
+                     max(version) AS version
+                   FROM wiki GROUP BY name"
+            deltaQuery="
+              SELECT DISTINCT
+                name
+              FROM
+                wiki
+              WHERE
+                time > (EXTRACT(
+                  epoch FROM TIMESTAMP '${dataimporter.wiki.last_index_time}'
+                ) * 1000000)"
+            deltaImportQuery="
+                   SELECT
+                     name,
+                     TIMESTAMP 'epoch' + (max(time)/1000000) * INTERVAL '1s' AS modified,
+                     max(version) AS version
+                   FROM 
+                     wiki
+                   WHERE
+                     name = '${dataimporter.delta.id}'
+                   GROUP BY name">
+      
+      <entity name="wiki_creation" 
+              dataSource="trac"
+              query="SELECT 
+                       author AS Creator,
+                       TIMESTAMP 'epoch' + (time/1000000) * INTERVAL '1s' AS created
+                     FROM 
+                       wiki 
+                     WHERE
+                       name = '${wiki.name}' 
+                       AND version = 1" />
+      
+      <entity name="wiki_collaborators"
+              dataSource="trac"
+              query="SELECT DISTINCT 
+                       author AS collaborator
+                     FROM 
+                       wiki 
+                     WHERE
+                       name = '${wiki.name}'
+                       AND author != ''" />
+      
+      <entity name="content"
+              dataSource="trac"
+              query="SELECT 
+                       text AS content 
+                     FROM
+                       wiki 
+                     WHERE 
+                       name = '${wiki.name}'
+                       AND version = '${wiki.version}'" />
+
+      <field column="UID" template="WIKI_${wiki.name}" />
+      <field column="getId" template="${wiki.name}" />
+      <field column="Type" template="wiki" />
+      <field column="Title" template="${wiki.name}" />
+      <field column="created" name="created" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+      <field column="modified" name="modified" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+      <field column="path_string" template="/wiki/${wiki.name}" />
+    </entity>
+
+    <entity name="ticket"
+            dataSource="trac"
+            transformer="TemplateTransformer,DateFormatTransformer"
+            pk="id"
+            deltaQuery="
+              SELECT
+                  id
+                FROM
+                  ticket
+              WHERE
+                time > (EXTRACT(
+                  epoch FROM TIMESTAMP '${dataimporter.ticket.last_index_time}'
+                ) * 1000000)"
+            query="SELECT
+                     id, 
+                     summary,
+                     description AS Description,
+                     milestone,
+                     priority,
+                     component,
+                     version,
+                     severity,
+                     reporter,
+                     owner,
+                     status,
+                     TIMESTAMP 'epoch' + (time/1000000)* INTERVAL '1s' AS created,
+                     TIMESTAMP 'epoch' + (changetime/1000000) * INTERVAL '1s' AS modified
+                   FROM
+                     ticket">
+
+      <entity name="ticket_collaborator"
+              dataSource="trac"
+              query="SELECT 
+                       reporter AS collaborator 
+                     FROM
+                       ticket
+                     WHERE 
+                       id = ${ticket.id}
+                     
+                     UNION
+                     
+                     SELECT 
+                       owner AS collaborator 
+                     FROM
+                       ticket
+                     WHERE 
+                       id = ${ticket.id}
+                     
+                     UNION
+                    
+                     SELECT DISTINCT
+                       author AS collaborator
+                     FROM 
+                       ticket_change 
+                     WHERE
+                       ticket = ${ticket.id}" />
+
+      <entity name="ticket_keywords"
+              dataSource="trac"
+              query="SELECT DISTINCT
+                       REGEXP_SPLIT_TO_TABLE(keywords, ',|\\s') AS keyword
+                     FROM 
+                       ticket
+                     WHERE
+                       id = ${ticket.id} AND
+                       keywords != ''" />
+      
+      <entity name="ticket_comments"
+              dataSource="trac"
+              query="SELECT
+                       newvalue AS comment
+                     FROM 
+                       ticket_change
+                     WHERE
+                       ticket = ${ticket.id}
+                       AND field = 'comment'" />
+
+      <field column="UID" template="TICKET_${ticket.id}" />
+      <field column="getId" template="${ticket.id}" />
+      <field column="Type" template="ticket" />
+      <field column="path_string" template="/ticket/${ticket.id}" />
+      <field column="Title" 
+             template="#${ticket.id} (${ticket.status}) - ${ticket.summary}" />
+      <field column="Creator" template="${ticket.reporter}" />
+      <field column="created" name="created" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+      <field column="modified" name="modified" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+    </entity>
+    
+    <entity name="changeset"
+            dataSource="trac"
+            transformer="TemplateTransformer,DateFormatTransformer"
+            pk="rev"
+            deltaQuery="
+            SELECT 
+              rev 
+            FROM 
+              revision 
+            WHERE
+              time > (EXTRACT(
+                epoch FROM TIMESTAMP '${dataimporter.changeset.last_index_time}'
+              ) * 1000000)"
+
+            query="SELECT
+                     rev AS revision, 
+                     author AS Creator,
+                     author AS collaborator,
+                     repos.value AS repos_name,
+                     TIMESTAMP 'epoch' + (time/1000000) * INTERVAL '1s' AS created,
+                     TIMESTAMP 'epoch' + (time/1000000) * INTERVAL '1s' AS modified,
+                     message                    
+                   FROM
+                     revision AS rev JOIN
+                     repository AS repos
+                       ON rev.repos = repos.id AND 
+                          repos.name = 'name' AND repos.value != ''">
+
+      <field column="UID" template="CHANGESET_${changeset.revision}" />
+      <field column="getId" template="${changeset.revision}" />
+      <field column="Type" template="changeset" />
+      <field column="path_string" 
+             template="/changeset/${changeset.revision}/${changeset.repos_name}"
+      />
+      <field column="Title" 
+             template="[${changeset.revision}] - ${changeset.message}" />
+      <field column="created" name="created" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+      <field column="modified" name="modified" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
+    </entity>
+
+    <entity name="thread"
+            dataSource="colab"
+            transformer="TemplateTransformer,DateFormatTransformer"
+            deltaQuery="
+              SELECT
+                thread_id AS id
+              FROM
+                super_archives_message
+              GROUP BY
+                thread_id
+              HAVING
+                max(received_time) > '${dataimporter.thread.last_index_time}'"
+            deltaImportQuery="SELECT
+                     sam.thread_id AS id,
+                     sat.subject_token AS name,
+                     sat.latest_message_id,
+                     saml.name AS mailinglist,
+                     array_to_string(array_agg(sam.body), ' ') AS content
+                   FROM
+                     super_archives_message AS sam
+                     JOIN super_archives_thread AS sat
+                       ON sat.id = sam.thread_id
+                     JOIN super_archives_mailinglist AS saml
+                       ON sat.mailinglist_id = saml.id
+                   WHERE
+                     sat.id = '${dataimporter.delta.id}'
+                   GROUP BY
+                     sam.thread_id,
+                     sat.subject_token,
+                     sat.latest_message_id,
+                     saml.name"
+
+            query="SELECT
+                     sam.thread_id AS id,
+                     sat.subject_token AS name,
+                     sat.latest_message_id,
+                     saml.name AS mailinglist,
+                     array_to_string(array_agg(sam.body), ' ') AS content
+                   FROM
+                     super_archives_message AS sam
+                     JOIN super_archives_thread AS sat
+                       ON sat.id = sam.thread_id
+                     JOIN super_archives_mailinglist AS saml
+                       ON sat.mailinglist_id = saml.id
+                   WHERE
+                     sat.spam IS NOT True
+                   GROUP BY
+                     sam.thread_id,
+                     sat.subject_token,
+                     sat.latest_message_id,
+                     saml.name">
+
+      <!--
+        Check about "DISTINCT ON" here: 
+          http://archives.postgresql.org/pgsql-general/2002-06/msg01330.php
+      -->
+      <entity name="first_message"
+              dataSource="colab"
+              transformer="TemplateTransformer"
+              query="SELECT DISTINCT ON (sam.thread_id)
+                       sam.body AS Description,
+                       sam.received_time AS created,
+                       sam.subject_clean AS subject,
+                       saea.real_name AS creator_real_name,
+                       saea.md5 AS creator_email_md5,
+                       au.username AS Creator
+                     FROM 
+                       super_archives_message AS sam
+                       JOIN super_archives_emailaddress AS saea
+                         ON sam.from_address_id = saea.id
+                       LEFT JOIN auth_user AS au
+                         ON au.id = saea.user_id
+                     WHERE
+                       sam.thread_id = ${thread.id}
+                     ORDER BY 
+                       sam.thread_id,
+                       sam.received_time">
+        <field column="Title" template="${first_message.subject}" />
+        <field column="creator_profile_uri" 
+               template="/user/hash/${first_message.creator_email_md5}" />
+      </entity>
+
+      <entity name="latest_message"
+              dataSource="colab"
+              query="SELECT 
+                       received_time AS modified
+                     FROM 
+                       super_archives_message
+                     WHERE
+                       id = ${thread.latest_message_id}" />
+
+      <entity name="thread_collaborators"
+              dataSource="colab"
+              query="SELECT DISTINCT 
+                       au.username AS collaborator
+                     FROM 
+                       super_archives_message AS sam
+                       JOIN super_archives_emailaddress AS saea
+                         ON sam.from_address_id = saea.id
+                       JOIN auth_user AS au
+                         ON au.id = saea.user_id
+                     WHERE
+                       thread_id = ${thread.id}" />
+
+      <field column="UID" template="THREAD_${thread.id}" />
+      <field column="getId" template="${thread.name}" />
+      <field column="Type" template="thread" />
+      <field column="path_string" template="/archives/thread/${thread.name}" />
+      <field column="created" name="created" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss" />
+      <field column="modified" name="modified" 
+             dateTimeFormat="yyyy-MM-dd hh:mm:ss" />
+    </entity> 
+  </document>
+          
+</dataConfig>
+  
+<!-- 
+vim: ts=2 sw=2 ss=2 expandtab:
+-->
+
@@ -0,0 +1,612 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<!--
+ This is the Solr schema file. This file should be named "schema.xml" and
+ should be in the conf directory under the solr home
+ (i.e. ./solr/conf/schema.xml by default)
+ or located where the classloader for the Solr webapp can find it.
+
+ This example schema is the recommended starting point for users.
+ It should be kept correct and concise, usable out-of-the-box.
+
+ For more information, on how to customize this file, please see
+ http://wiki.apache.org/solr/SchemaXml
+-->
+
+<schema name="solr-instance" version="1.4">
+  <!-- attribute "name" is the name of this schema and is only used for display purposes.
+       Applications should change this to reflect the nature of the search collection.
+       version="1.4" is Solr's version number for the schema syntax and semantics.  It should
+       not normally be changed by applications.
+       1.0: multiValued attribute did not exist, all fields are multiValued by nature
+       1.1: multiValued attribute introduced, false by default
+       1.2: omitTermFreqAndPositions attribute introduced, true by default except for text fields.
+       1.3: removed optional field compress feature
+       1.4: default auto-phrase (QueryParser feature) to off
+     -->
+
+  <types>
+    <!-- field type definitions. The "name" attribute is
+       just a label to be used by field definitions.  The "class"
+       attribute and any other attributes determine the real
+       behavior of the fieldType.
+         Class names starting with "solr" refer to java classes in the
+       org.apache.solr.analysis package.
+    -->
+
+    <!-- The StrField type is not analyzed, but indexed/stored verbatim. -->
+    <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
+
+    <!-- boolean type: "true" or "false" -->
+    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
+    <!--Binary data type. The data should be sent/retrieved in as Base64 encoded Strings -->
+    <fieldtype name="binary" class="solr.BinaryField"/>
+
+    <!-- The optional sortMissingLast and sortMissingFirst attributes are
+         currently supported on types that are sorted internally as strings.
+	       This includes "string","boolean","sint","slong","sfloat","sdouble","pdate"
+       - If sortMissingLast="true", then a sort on this field will cause documents
+         without the field to come after documents with the field,
+         regardless of the requested sort order (asc or desc).
+       - If sortMissingFirst="true", then a sort on this field will cause documents
+         without the field to come before documents with the field,
+         regardless of the requested sort order.
+       - If sortMissingLast="false" and sortMissingFirst="false" (the default),
+         then default lucene sorting will be used which places docs without the
+         field first in an ascending sort and last in a descending sort.
+    -->
+
+    <!--
+      Default numeric field types. For faster range queries, consider the tint/tfloat/tlong/tdouble types.
+    -->
+    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+
+    <!-- BBB support for existing schemas based on collective.solr -->
+    <fieldType name="integer" class="solr.TrieIntField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+
+    <!--
+     Numeric field types that index each value at various levels of precision
+     to accelerate range queries when the number of values between the range
+     endpoints is large. See the javadoc for NumericRangeQuery for internal
+     implementation details.
+
+     Smaller precisionStep values (specified in bits) will lead to more tokens
+     indexed per value, slightly larger index size, and faster range queries.
+     A precisionStep of 0 disables indexing at different precision levels.
+    -->
+    <fieldType name="tint" class="solr.TrieIntField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tlong" class="solr.TrieLongField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+
+    <!-- The format for this date field is of the form 1995-12-31T23:59:59Z, and
+         is a more restricted form of the canonical representation of dateTime
+         http://www.w3.org/TR/xmlschema-2/#dateTime
+         The trailing "Z" designates UTC time and is mandatory.
+         Optional fractional seconds are allowed: 1995-12-31T23:59:59.999Z
+         All other components are mandatory.
+
+         Expressions can also be used to denote calculations that should be
+         performed relative to "NOW" to determine the value, ie...
+
+               NOW/HOUR
+                  ... Round to the start of the current hour
+               NOW-1DAY
+                  ... Exactly 1 day prior to now
+               NOW/DAY+6MONTHS+3DAYS
+                  ... 6 months and 3 days in the future from the start of
+                      the current day
+
+         Consult the DateField javadocs for more information.
+
+         Note: For faster range queries, consider the tdate type
+      -->
+    <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
+
+    <!-- A Trie based date field for faster date range queries and date faceting. -->
+    <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
+
+    <!-- The "RandomSortField" is not used to store or search any
+         data.  You can declare fields of this type it in your schema
+         to generate pseudo-random orderings of your docs for sorting
+         purposes.  The ordering is generated based on the field name
+         and the version of the index, As long as the index version
+         remains unchanged, and the same field name is reused,
+         the ordering of the docs will be consistent.
+         If you want different psuedo-random orderings of documents,
+         for the same version of the index, use a dynamicField and
+         change the name
+     -->
+    <fieldType name="random" class="solr.RandomSortField" indexed="true" />
+
+    <!-- solr.TextField allows the specification of custom text analyzers
+         specified as a tokenizer and a list of token filters. Different
+         analyzers may be specified for indexing and querying.
+
+         The optional positionIncrementGap puts space between multiple fields of
+         this type on the same document, with the purpose of preventing false phrase
+         matching across fields.
+
+         For more info on customizing your analyzer chain, please see
+         http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
+     -->
+
+    <!-- One can also specify an existing Analyzer class that has a
+         default constructor via the class attribute on the analyzer element
+    <fieldType name="text_greek" class="solr.TextField">
+      <analyzer class="org.apache.lucene.analysis.el.GreekAnalyzer"/>
+    </fieldType>
+    -->
+    
+    <fieldType name="text_ptbr" class="solr.TextField">
+      <analyzer class="org.apache.lucene.analysis.br.BrazilianAnalyzer">
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.BrazilianStemFilterFactory"/>
+      </analyzer>
+    </fieldType>
+    
+    <!-- A text field that only splits on whitespace for exact matching of words -->
+    <fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- A general text field that has reasonable, generic
+         cross-language defaults: it tokenizes with StandardTokenizer,
+	 removes stop words from case-insensitive "stopwords.txt"
+	 (empty by default), and down cases.  At query time only, it
+	 also applies synonyms. -->
+    <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+        <!-- in this example, we will only use synonyms at query time
+        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
+        -->
+        <filter class="solr.LowerCaseFilterFactory"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- A text field with defaults appropriate for English: it
+         tokenizes with StandardTokenizer, removes English stop words
+         (stopwords_en.txt), down cases, protects words from protwords.txt, and
+         finally applies Porter's stemming.  The query time analyzer
+         also applies synonyms from synonyms.txt. -->
+    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <!-- in this example, we will only use synonyms at query time
+        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
+        -->
+        <!-- Case insensitive stop word removal.
+          add enablePositionIncrements=true in both the index and query
+          analyzers to leave a 'gap' for more accurate phrase queries.
+        -->
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords_en.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+	<filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+	<!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+        <filter class="solr.EnglishMinimalStemFilterFactory"/>
+	-->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords_en.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+	<filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+	<!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+        <filter class="solr.EnglishMinimalStemFilterFactory"/>
+	-->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- A text field with defaults appropriate for English, plus
+	 aggressive word-splitting and autophrase features enabled.
+	 This field is just like text_en, except it adds
+	 WordDelimiterFilter to enable splitting and matching of
+	 words on case-change, alpha numeric boundaries, and
+	 non-alphanumeric chars.  This means certain compound word
+	 cases will work, for example query "wi fi" will match
+	 document "WiFi" or "wi-fi".  However, other cases will still
+	 not match, for example if the query is "wifi" and the
+	 document is "wi fi" or if the query is "wi-fi" and the
+	 document is "wifi".
+        -->
+    <fieldType name="text_en_splitting" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
+      <analyzer type="index">
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+        <!-- in this example, we will only use synonyms at query time
+        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
+        -->
+        <!-- Case insensitive stop word removal.
+          add enablePositionIncrements=true in both the index and query
+          analyzers to leave a 'gap' for more accurate phrase queries.
+        -->
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords_en.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords_en.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- Less flexible matching, but less false matches.  Probably not ideal for product names,
+         but may be good for SKUs.  Can insert dashes in the wrong place and still match. -->
+    <fieldType name="text_en_splitting_tight" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
+      <analyzer>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_en.txt"/>
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="0" generateNumberParts="0" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <filter class="solr.EnglishMinimalStemFilterFactory"/>
+        <!-- this filter can remove any duplicate tokens that appear at the same position - sometimes
+             possible with WordDelimiterFilter in conjuncton with stemming. -->
+        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- Just like text_general except it reverses the characters of
+	 each token, to enable more efficient leading wildcard queries. -->
+    <fieldType name="text_general_rev" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.ReversedWildcardFilterFactory" withOriginal="true"
+           maxPosAsterisk="3" maxPosQuestion="2" maxFractionAsterisk="0.33"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+        <filter class="solr.LowerCaseFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- charFilter + WhitespaceTokenizer  -->
+    <!--
+    <fieldType name="text_char_norm" class="solr.TextField" positionIncrementGap="100" >
+      <analyzer>
+        <charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+      </analyzer>
+    </fieldType>
+    -->
+
+    <!-- This is an example of using the KeywordTokenizer along
+         With various TokenFilterFactories to produce a sortable field
+         that does not include some properties of the source text
+      -->
+    <fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
+      <analyzer>
+        <!-- KeywordTokenizer does no actual tokenizing, so the entire
+             input string is preserved as a single token
+          -->
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <!-- The LowerCase TokenFilter does what you expect, which can be
+             when you want your sorting to be case insensitive
+          -->
+        <filter class="solr.LowerCaseFilterFactory" />
+        <!-- The TrimFilter removes any leading or trailing whitespace -->
+        <filter class="solr.TrimFilterFactory" />
+        <!-- The PatternReplaceFilter gives you the flexibility to use
+             Java Regular expression to replace any sequence of characters
+             matching a pattern with an arbitrary replacement string, 
+             which may include back references to portions of the original
+             string matched by the pattern.
+             
+             See the Java Regular Expression documentation for more
+             information on pattern and replacement string syntax.
+             
+             http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.html
+          -->
+        <filter class="solr.PatternReplaceFilterFactory"
+                pattern="([^a-z])" replacement="" replace="all"
+        />
+      </analyzer>
+    </fieldType>
+    
+    <fieldtype name="phonetic" stored="false" indexed="true" class="solr.TextField" >
+      <analyzer>
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.DoubleMetaphoneFilterFactory" inject="false"/>
+      </analyzer>
+    </fieldtype>
+
+    <fieldtype name="payloads" stored="false" indexed="true" class="solr.TextField" >
+      <analyzer>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+        <!--
+        The DelimitedPayloadTokenFilter can put payloads on tokens... for example,
+        a token of "foo|1.4"  would be indexed as "foo" with a payload of 1.4f
+        Attributes of the DelimitedPayloadTokenFilterFactory : 
+         "delimiter" - a one character delimiter. Default is | (pipe)
+	 "encoder" - how to encode the following value into a playload
+	    float -> org.apache.lucene.analysis.payloads.FloatEncoder,
+	    integer -> o.a.l.a.p.IntegerEncoder
+	    identity -> o.a.l.a.p.IdentityEncoder
+            Fully Qualified class name implementing PayloadEncoder, Encoder must have a no arg constructor.
+         -->
+        <filter class="solr.DelimitedPayloadTokenFilterFactory" encoder="float"/>
+      </analyzer>
+    </fieldtype>
+
+    <!-- lowercases the entire field value, keeping it as a single token.  -->
+    <fieldType name="lowercase" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory" />
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="text_path" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.PathHierarchyTokenizerFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <!-- since fields of this type are by default not stored or indexed,
+         any data added to them will be ignored outright.  -->
+    <fieldtype name="ignored" stored="false" indexed="false" multiValued="true" class="solr.StrField" />
+
+    <!-- This point type indexes the coordinates as separate fields (subFields)
+      If subFieldType is defined, it references a type, and a dynamic field
+      definition is created matching *___<typename>.  Alternately, if
+      subFieldSuffix is defined, that is used to create the subFields.
+      Example: if subFieldType="double", then the coordinates would be
+        indexed in fields myloc_0___double,myloc_1___double.
+      Example: if subFieldSuffix="_d" then the coordinates would be indexed
+        in fields myloc_0_d,myloc_1_d
+      The subFields are an implementation detail of the fieldType, and end
+      users normally should not need to know about them.
+     -->
+    <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
+
+    <!-- A specialized field for geospatial search. If indexed, this fieldType must not be multivalued. -->
+    <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
+
+   <!--
+    A Geohash is a compact representation of a latitude longitude pair in a single field.
+    See http://wiki.apache.org/solr/SpatialSearch
+   -->
+    <fieldtype name="geohash" class="solr.GeoHashField"/>
+
+
+ </types>
+
+
+ <fields>
+   <!-- Valid attributes for fields:
+     name: mandatory - the name for the field
+     type: mandatory - the name of a previously defined type from the
+       <types> section
+     indexed: true if this field should be indexed (searchable or sortable)
+     stored: true if this field should be retrievable
+     multiValued: true if this field may contain multiple values per document
+     omitNorms: (expert) set to true to omit the norms associated with
+       this field (this disables length normalization and index-time
+       boosting for the field, and saves some memory).  Only full-text
+       fields or fields that need an index-time boost need norms.
+     termVectors: [false] set to true to store the term vector for a
+       given field.
+       When using MoreLikeThis, fields used for similarity should be
+       stored for best performance.
+     termPositions: Store position information with the term vector.
+       This will increase storage costs.
+     termOffsets: Store offset information with the term vector. This
+       will increase storage costs.
+     default: a value that should be used if no value is specified
+       when adding a document.
+   -->
+   
+   <!--  Base fields (all should be indexed and stored)-->
+   <field name="UID" type="string" indexed="true"
+          stored="true" required="true" multiValued="false"/>
+   <field name="getId" type="string" indexed="false"
+          stored="true" required="true" multiValued="false"/>
+   <field name="Type" type="string" indexed="true"
+          stored="true" required="true" multiValued="false"/>
+   <field name="Title" type="text_ptbr" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="Description" type="text_ptbr" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="Creator" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="created" type="date" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="modified" type="date" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="mailinglist" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="creator_real_name" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="creator_profile_uri" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+          
+   <!-- All next fields shoult NOT be stored --> 
+   <field name="name" type="string" indexed="true" 
+          stored="false" required="false" multiValued="false"/>
+   <field name="comment" type="text_ptbr" indexed="true" 
+          stored="false" required="false" multiValued="true" />
+   <field name="content" type="text_ptbr" indexed="true" 
+          stored="false" required="false" multiValued="false"/>
+   <field name="keyword" type="text_ptbr" indexed="true" 
+          stored="false" required="false" multiValued="true"/>
+   <field name="milestone" type="string" indexed="true" 
+          stored="false" required="false" multiValued="false"/>
+   <field name="priority" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="component" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="version" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="severity" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="reporter" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="owner" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="status" type="string" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="revision" type="int" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="subject" type="text_ptbr" indexed="true" 
+          stored="false" required="false" multiValued="false"/>
+   <field name="path_string" type="string" indexed="false"
+          stored="true" required="false" multiValued="false"/>
+
+   <!-- Plone only fields -->
+   <field name="review_state" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="effective" type="date" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="expires" type="date" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="getIcon" type="string" indexed="false"
+          stored="true" required="false" multiValued="false"/>
+   <field name="getRemoteUrl" type="string" indexed="false"
+          stored="true" required="false" multiValued="false"/>
+   <field name="exclude_from_nav" type="boolean" indexed="false"
+          stored="true" required="false" multiValued="false"/>
+   <field name="portal_type" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="is_folderish" type="boolean" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="Language" type="string" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="Date" type="date" indexed="true"
+          stored="true" required="false" multiValued="false"/>
+   <field name="allowedRolesAndUsers" type="string" indexed="true"
+          stored="false" required="false" multiValued="true"/>
+   <field name="object_provides" type="string" indexed="true"
+          stored="false" required="false" multiValued="true"/>
+   <field name="path_depth" type="integer" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="path_parents" type="string" indexed="true"
+          stored="false" required="false" multiValued="true"/>
+   <field name="SearchableText" type="text_general" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+   <field name="searchwords" type="string" indexed="true"
+          stored="false" required="false" multiValued="true"/>
+   <field name="showinsearch" type="boolean" indexed="true"
+          stored="false" required="false" multiValued="false"/>
+
+   <dynamicField name="tika_*" type="ignored" />
+  
+   <field name="collaborator" type="string" indexed="true" 
+          stored="false" required="false" multiValued="true" /> 
+
+   <!-- catchall field, containing all other searchable text fields (implemented
+        via copyField further on in this schema  -->
+   <field name="default" type="text_general" indexed="true"
+          stored="false" required="false" multiValued="true"/>
+ </fields>
+
+ <!-- Field to use to determine and enforce document uniqueness. 
+      Unless this field is marked with required="false", it will be a required field
+   -->
+ <uniqueKey>UID</uniqueKey>
+
+ <!-- field for the QueryParser to use when an explicit fieldname is absent -->
+ <defaultSearchField>default</defaultSearchField>
+
+ <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
+ <solrQueryParser defaultOperator="OR"/>
+
+  <!-- copyField commands copy one field to another at the time a document
+        is added to the index.  It's used either to index the same field differently,
+        or to add multiple fields to the same field for easier/faster searching.  -->
+   <copyField source="Description" dest="default"/>
+   <copyField source="SearchableText" dest="default"/>
+   <copyField source="Title" dest="default"/>
+   <copyField source="content" dest="default" />
+   <copyField source="Creator" dest="default" />
+   <copyField source="collaborator" dest="default" />
+   <copyField source="comment" dest="default" />
+   <copyField source="keyword" dest="default" />
+
+   <!-- Above, multiple source fields are copied to the [text] field. 
+	  Another way to map multiple source fields to the same 
+	  destination field is to use the dynamic field syntax. 
+	  copyField also supports a maxChars to copy setting.  -->
+	   
+   <!-- <copyField source="*_t" dest="text" maxChars="3000"/> -->
+
+   <!-- copy name to alphaNameSort, a field designed for sorting by name -->
+   <!-- <copyField source="name" dest="alphaNameSort"/> -->
+
+ <!-- Similarity is the scoring routine for each document vs. a query.
+      A custom similarity may be specified here, but the default is fine
+      for most applications.  -->
+ <!-- <similarity class="org.apache.lucene.search.DefaultSimilarity"/> -->
+ <!-- ... OR ...
+      Specify a SimilarityFactory class name implementation
+      allowing parameters to be used.
+ -->
+ <!--
+ <similarity class="com.example.solr.CustomSimilarityFactory">
+   <str name="paramkey">param value</str>
+ </similarity>
+ -->
+
+
+</schema>
+
@@ -0,0 +1,12 @@
+<!--
+    Context configuration file for the Solr Web App
+-->
+
+<Context path="/solr" docBase="/usr/local/share/solr/apache-solr-3.3.0.war"
+   debug="0" privileged="true" allowLinking="true" crossContext="true">
+     <!-- make symlinks work in Tomcat -->
+  <Resources className="org.apache.naming.resources.FileDirContext" allowLinking="true" />
+
+  <Environment name="solr/home" type="java.lang.String" value="/usr/local/share/solr" override="true" />
+  </Context>
+
@@ -0,0 +1,1535 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<!-- 
+     For more details about configurations options that may appear in
+     this file, see http://wiki.apache.org/solr/SolrConfigXml. 
+-->
+<config>
+  <!-- In all configuration below, a prefix of "solr." for class names
+       is an alias that causes solr to search appropriate packages,
+       including org.apache.solr.(search|update|request|core|analysis)
+
+       You may also specify a fully qualified Java classname if you
+       have your own custom plugins.
+    -->
+
+  <!-- Set this to 'false' if you want solr to continue working after
+       it has encountered an severe configuration error.  In a
+       production environment, you may want solr to keep working even
+       if one handler is mis-configured.
+
+       You may also set this to false using by setting the system
+       property:
+
+         -Dsolr.abortOnConfigurationError=false
+    -->
+  <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
+  
+  <!-- Controls what version of Lucene various components of Solr
+       adhere to.  Generally, you want to use the latest version to
+       get all bug fixes and improvements. It is highly recommended
+       that you fully re-index after changing this setting as it can
+       affect both how text is indexed and queried.
+    -->
+  <luceneMatchVersion>LUCENE_33</luceneMatchVersion>
+
+  <!-- lib directives can be used to instruct Solr to load an Jars
+       identified and use them to resolve any "plugins" specified in
+       your solrconfig.xml or schema.xml (ie: Analyzers, Request
+       Handlers, etc...).
+
+       All directories and paths are resolved relative to the
+       instanceDir.
+
+       If a "./lib" directory exists in your instanceDir, all files
+       found in it are included as if you had used the following
+       syntax...
+       
+              <lib dir="./lib" />
+    -->
+  <!-- A dir option by itself adds any files found in the directory to
+       the classpath, this is useful for including all jars in a
+       directory.
+    -->
+  <lib dir="../../contrib/extraction/lib" />
+  <!-- When a regex is specified in addition to a directory, only the
+       files in that directory which completely match the regex
+       (anchored on both ends) will be included.
+    -->
+  <lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
+  <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" />
+  <lib dir="../../dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />
+
+  <!-- If a dir option (with or without a regex) is used and nothing
+       is found that matches, it will be ignored
+    -->
+  <lib dir="../../contrib/clustering/lib/" />
+  <lib dir="/total/crap/dir/ignored" /> 
+  <!-- an exact path can be used to specify a specific file.  This
+       will cause a serious error to be logged if it can't be loaded.
+    -->
+  <!--
+  <lib path="../a-jar-that-does-not-exist.jar" /> 
+  -->
+  
+  <!-- Data Directory
+
+       Used to specify an alternate directory to hold all index data
+       other than the default ./data under the Solr home.  If
+       replication is in use, this should match the replication
+       configuration.
+    -->
+  <dataDir>${solr.data.dir:/var/local/lib/solr/data}</dataDir>
+
+
+  <!-- The DirectoryFactory to use for indexes.
+       
+       solr.StandardDirectoryFactory, the default, is filesystem
+       based.  solr.RAMDirectoryFactory is memory based, not
+       persistent, and doesn't work with replication.
+    -->
+  <directoryFactory name="DirectoryFactory" 
+                    class="${solr.directoryFactory:solr.StandardDirectoryFactory}"/>
+
+
+  <!-- Index Defaults
+
+       Values here affect all index writers and act as a default
+       unless overridden.
+
+       WARNING: See also the <mainIndex> section below for parameters
+       that overfor Solr's main Lucene index.
+    -->
+  <indexDefaults>
+
+    <useCompoundFile>false</useCompoundFile>
+
+    <mergeFactor>10</mergeFactor>
+    <!-- Sets the amount of RAM that may be used by Lucene indexing
+         for buffering added documents and deletions before they are
+         flushed to the Directory.  -->
+    <ramBufferSizeMB>32</ramBufferSizeMB>
+    <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then
+         Lucene will flush based on whichever limit is hit first.  
+      -->
+    <!-- <maxBufferedDocs>1000</maxBufferedDocs> -->
+
+    <maxFieldLength>10000</maxFieldLength>
+    <writeLockTimeout>1000</writeLockTimeout>
+    <commitLockTimeout>10000</commitLockTimeout>
+
+    <!-- Expert: Merge Policy 
+
+         The Merge Policy in Lucene controls how merging is handled by
+         Lucene.  The default in Solr 3.3 is TieredMergePolicy.
+         
+         The default in 2.3 was the LogByteSizeMergePolicy,
+         previous versions used LogDocMergePolicy.
+         
+         LogByteSizeMergePolicy chooses segments to merge based on
+         their size.  The Lucene 2.2 default, LogDocMergePolicy chose
+         when to merge based on number of documents
+         
+         Other implementations of MergePolicy must have a no-argument
+         constructor
+      -->
+    <!--
+       <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"/>
+       -->
+
+    <!-- Expert: Merge Scheduler
+
+         The Merge Scheduler in Lucene controls how merges are
+         performed.  The ConcurrentMergeScheduler (Lucene 2.3 default)
+         can perform merges in the background using separate threads.
+         The SerialMergeScheduler (Lucene 2.2 default) does not.
+     -->
+    <!-- 
+       <mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>
+       -->
+	  
+    <!-- LockFactory 
+
+         This option specifies which Lucene LockFactory implementation
+         to use.
+      
+         single = SingleInstanceLockFactory - suggested for a
+                  read-only index or when there is no possibility of
+                  another process trying to modify the index.
+         native = NativeFSLockFactory - uses OS native file locking.
+                  Do not use when multiple solr webapps in the same
+                  JVM are attempting to share a single index.
+         simple = SimpleFSLockFactory  - uses a plain file for locking
+
+         (For backwards compatibility with Solr 1.2, 'simple' is the
+         default if not specified.)
+
+         More details on the nuances of each LockFactory...
+         http://wiki.apache.org/lucene-java/AvailableLockFactories
+    -->
+    <lockType>native</lockType>
+
+    <!-- Expert: Controls how often Lucene loads terms into memory
+         Default is 128 and is likely good for most everyone.
+      -->
+    <!-- <termIndexInterval>256</termIndexInterval> -->
+  </indexDefaults>
+
+  <!-- Main Index
+
+       Values here override the values in the <indexDefaults> section
+       for the main on disk index.
+    -->
+  <mainIndex>
+
+    <useCompoundFile>false</useCompoundFile>
+    <ramBufferSizeMB>32</ramBufferSizeMB>
+    <mergeFactor>10</mergeFactor>
+
+    <!-- Unlock On Startup
+
+         If true, unlock any held write or commit locks on startup.
+         This defeats the locking mechanism that allows multiple
+         processes to safely access a lucene index, and should be used
+         with care.
+
+         This is not needed if lock type is 'none' or 'single'
+     -->
+    <unlockOnStartup>false</unlockOnStartup>
+    
+    <!-- If true, IndexReaders will be reopened (often more efficient)
+         instead of closed and then opened.
+      -->
+    <reopenReaders>true</reopenReaders>
+
+    <!-- Commit Deletion Policy
+
+         Custom deletion policies can specified here. The class must
+         implement org.apache.lucene.index.IndexDeletionPolicy.
+
+         http://lucene.apache.org/java/2_9_1/api/all/org/apache/lucene/index/IndexDeletionPolicy.html
+
+         The standard Solr IndexDeletionPolicy implementation supports
+         deleting index commit points on number of commits, age of
+         commit point and optimized status.
+         
+         The latest commit point should always be preserved regardless
+         of the criteria.
+    -->
+    <deletionPolicy class="solr.SolrDeletionPolicy">
+      <!-- The number of commit points to be kept -->
+      <str name="maxCommitsToKeep">1</str>
+      <!-- The number of optimized commit points to be kept -->
+      <str name="maxOptimizedCommitsToKeep">0</str>
+      <!--
+          Delete all commit points once they have reached the given age.
+          Supports DateMathParser syntax e.g.
+        -->
+      <!--
+         <str name="maxCommitAge">30MINUTES</str>
+         <str name="maxCommitAge">1DAY</str>
+      -->
+    </deletionPolicy>
+
+    <!-- Lucene Infostream
+       
+         To aid in advanced debugging, Lucene provides an "InfoStream"
+         of detailed information when indexing.
+
+         Setting The value to true will instruct the underlying Lucene
+         IndexWriter to write its debugging info the specified file
+      -->
+     <infoStream file="INFOSTREAM.txt">false</infoStream> 
+
+  </mainIndex>
+
+  <!-- JMX
+       
+       This example enables JMX if and only if an existing MBeanServer
+       is found, use this if you want to configure JMX through JVM
+       parameters. Remove this to disable exposing Solr configuration
+       and statistics to JMX.
+
+       For more details see http://wiki.apache.org/solr/SolrJmx
+    -->
+  <jmx />
+  <!-- If you want to connect to a particular server, specify the
+       agentId 
+    -->
+  <!-- <jmx agentId="myAgent" /> -->
+  <!-- If you want to start a new MBeanServer, specify the serviceUrl -->
+  <!-- <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
+    -->
+
+  <!-- The default high-performance update handler -->
+  <updateHandler class="solr.DirectUpdateHandler2">
+
+    <!-- AutoCommit
+
+         Perform a <commit/> automatically under certain conditions.
+         Instead of enabling autoCommit, consider using "commitWithin"
+         when adding documents. 
+
+         http://wiki.apache.org/solr/UpdateXmlMessages
+
+         maxDocs - Maximum number of documents to add since the last
+                   commit before automatically triggering a new commit.
+
+         maxTime - Maximum amount of time that is allowed to pass
+                   since a document was added before automaticly
+                   triggering a new commit. 
+      -->
+    <!--
+       <autoCommit> 
+         <maxDocs>10000</maxDocs>
+         <maxTime>1000</maxTime> 
+       </autoCommit>
+      -->
+
+    <!-- Update Related Event Listeners
+         
+         Various IndexWriter related events can trigger Listeners to
+         take actions.
+
+         postCommit - fired after every commit or optimize command
+         postOptimize - fired after every optimize command
+      -->
+    <!-- The RunExecutableListener executes an external command from a
+         hook such as postCommit or postOptimize.
+         
+         exe - the name of the executable to run
+         dir - dir to use as the current working directory. (default=".")
+         wait - the calling thread waits until the executable returns. 
+                (default="true")
+         args - the arguments to pass to the program.  (default is none)
+         env - environment variables to set.  (default is none)
+      -->
+    <!-- This example shows how RunExecutableListener could be used
+         with the script based replication...
+         http://wiki.apache.org/solr/CollectionDistribution
+      -->
+    <!--
+       <listener event="postCommit" class="solr.RunExecutableListener">
+         <str name="exe">solr/bin/snapshooter</str>
+         <str name="dir">.</str>
+         <bool name="wait">true</bool>
+         <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
+         <arr name="env"> <str>MYVAR=val1</str> </arr>
+       </listener>
+      -->
+  </updateHandler>
+  
+  <!-- IndexReaderFactory
+
+       Use the following format to specify a custom IndexReaderFactory,
+       which allows for alternate IndexReader implementations.
+
+       ** Experimental Feature **
+
+       Please note - Using a custom IndexReaderFactory may prevent
+       certain other features from working. The API to
+       IndexReaderFactory may change without warning or may even be
+       removed from future releases if the problems cannot be
+       resolved.
+
+
+       ** Features that may not work with custom IndexReaderFactory **
+
+       The ReplicationHandler assumes a disk-resident index. Using a
+       custom IndexReader implementation may cause incompatibility
+       with ReplicationHandler and may cause replication to not work
+       correctly. See SOLR-1366 for details.
+
+    -->
+  <!--
+  <indexReaderFactory name="IndexReaderFactory" class="package.class">
+    <str name="someArg">Some Value</str>
+  </indexReaderFactory >
+  -->
+  <!-- By explicitly declaring the Factory, the termIndexDivisor can
+       be specified.
+    -->
+  <!--
+     <indexReaderFactory name="IndexReaderFactory" 
+                         class="solr.StandardIndexReaderFactory">
+       <int name="setTermIndexDivisor">12</int>
+     </indexReaderFactory >
+    -->
+
+
+  <query>
+    <!-- Max Boolean Clauses
+
+         Maximum number of clauses in each BooleanQuery,  an exception
+         is thrown if exceeded.
+
+         ** WARNING **
+         
+         This option actually modifies a global Lucene property that
+         will affect all SolrCores.  If multiple solrconfig.xml files
+         disagree on this property, the value at any given moment will
+         be based on the last SolrCore to be initialized.
+         
+      -->
+    <maxBooleanClauses>1024</maxBooleanClauses>
+
+
+    <!-- Solr Internal Query Caches
+
+         There are two implementations of cache available for Solr,
+         LRUCache, based on a synchronized LinkedHashMap, and
+         FastLRUCache, based on a ConcurrentHashMap.  
+
+         FastLRUCache has faster gets and slower puts in single
+         threaded operation and thus is generally faster than LRUCache
+         when the hit ratio of the cache is high (> 75%), and may be
+         faster under other scenarios on multi-cpu systems.
+    -->
+
+    <!-- Filter Cache
+
+         Cache used by SolrIndexSearcher for filters (DocSets),
+         unordered sets of *all* documents that match a query.  When a
+         new searcher is opened, its caches may be prepopulated or
+         "autowarmed" using data from caches in the old searcher.
+         autowarmCount is the number of items to prepopulate.  For
+         LRUCache, the autowarmed items will be the most recently
+         accessed items.
+
+         Parameters:
+           class - the SolrCache implementation LRUCache or
+               (LRUCache or FastLRUCache)
+           size - the maximum number of entries in the cache
+           initialSize - the initial capacity (number of entries) of
+               the cache.  (see java.util.HashMap)
+           autowarmCount - the number of entries to prepopulate from
+               and old cache.  
+      -->
+    <filterCache class="solr.FastLRUCache"
+                 size="512"
+                 initialSize="512"
+                 autowarmCount="0"/>
+
+    <!-- Query Result Cache
+         
+         Caches results of searches - ordered lists of document ids
+         (DocList) based on a query, a sort, and the range of documents requested.  
+      -->
+    <queryResultCache class="solr.LRUCache"
+                     size="512"
+                     initialSize="512"
+                     autowarmCount="0"/>
+   
+    <!-- Document Cache
+
+         Caches Lucene Document objects (the stored fields for each
+         document).  Since Lucene internal document ids are transient,
+         this cache will not be autowarmed.  
+      -->
+    <documentCache class="solr.LRUCache"
+                   size="512"
+                   initialSize="512"
+                   autowarmCount="0"/>
+    
+    <!-- Field Value Cache
+         
+         Cache used to hold field values that are quickly accessible
+         by document id.  The fieldValueCache is created by default
+         even if not configured here.
+      -->
+    <!--
+       <fieldValueCache class="solr.FastLRUCache"
+                        size="512"
+                        autowarmCount="128"
+                        showItems="32" />
+      -->
+
+    <!-- Custom Cache
+
+         Example of a generic cache.  These caches may be accessed by
+         name through SolrIndexSearcher.getCache(),cacheLookup(), and
+         cacheInsert().  The purpose is to enable easy caching of
+         user/application level data.  The regenerator argument should
+         be specified as an implementation of solr.CacheRegenerator 
+         if autowarming is desired.  
+      -->
+    <!--
+       <cache name="myUserCache"
+              class="solr.LRUCache"
+              size="4096"
+              initialSize="1024"
+              autowarmCount="1024"
+              regenerator="com.mycompany.MyRegenerator"
+              />
+      -->
+
+
+    <!-- Lazy Field Loading
+
+         If true, stored fields that are not requested will be loaded
+         lazily.  This can result in a significant speed improvement
+         if the usual case is to not load all stored fields,
+         especially if the skipped fields are large compressed text
+         fields.
+    -->
+    <enableLazyFieldLoading>true</enableLazyFieldLoading>
+
+   <!-- Use Filter For Sorted Query
+
+        A possible optimization that attempts to use a filter to
+        satisfy a search.  If the requested sort does not include
+        score, then the filterCache will be checked for a filter
+        matching the query. If found, the filter will be used as the
+        source of document ids, and then the sort will be applied to
+        that.
+
+        For most situations, this will not be useful unless you
+        frequently get the same search repeatedly with different sort
+        options, and none of them ever use "score"
+     -->
+   <!--
+      <useFilterForSortedQuery>true</useFilterForSortedQuery>
+     -->
+
+   <!-- Result Window Size
+
+        An optimization for use with the queryResultCache.  When a search
+        is requested, a superset of the requested number of document ids
+        are collected.  For example, if a search for a particular query
+        requests matching documents 10 through 19, and queryWindowSize is 50,
+        then documents 0 through 49 will be collected and cached.  Any further
+        requests in that range can be satisfied via the cache.  
+     -->
+   <queryResultWindowSize>20</queryResultWindowSize>
+
+   <!-- Maximum number of documents to cache for any entry in the
+        queryResultCache. 
+     -->
+   <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
+
+   <!-- Query Related Event Listeners
+
+        Various IndexSearcher related events can trigger Listeners to
+        take actions.
+
+        newSearcher - fired whenever a new searcher is being prepared
+        and there is a current searcher handling requests (aka
+        registered).  It can be used to prime certain caches to
+        prevent long request times for certain requests.
+
+        firstSearcher - fired whenever a new searcher is being
+        prepared but there is no current registered searcher to handle
+        requests or to gain autowarming data from.
+
+        
+     -->
+    <!-- QuerySenderListener takes an array of NamedList and executes a
+         local query request for each NamedList in sequence. 
+      -->
+    <listener event="newSearcher" class="solr.QuerySenderListener">
+      <arr name="queries">
+        <!--
+           <lst><str name="q">solr</str><str name="sort">price asc</str></lst>
+           <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>
+          -->
+      </arr>
+    </listener>
+    <listener event="firstSearcher" class="solr.QuerySenderListener">
+      <arr name="queries">
+        <lst>
+          <str name="q">static firstSearcher warming in solrconfig.xml</str>
+        </lst>
+      </arr>
+    </listener>
+
+    <!-- Use Cold Searcher
+
+         If a search request comes in and there is no current
+         registered searcher, then immediately register the still
+         warming searcher and use it.  If "false" then all requests
+         will block until the first searcher is done warming.
+      -->
+    <useColdSearcher>false</useColdSearcher>
+
+    <!-- Max Warming Searchers
+         
+         Maximum number of searchers that may be warming in the
+         background concurrently.  An error is returned if this limit
+         is exceeded.
+
+         Recommend values of 1-2 for read-only slaves, higher for
+         masters w/o cache warming.
+      -->
+    <maxWarmingSearchers>2</maxWarmingSearchers>
+
+  </query>
+
+
+  <!-- Request Dispatcher
+
+       This section contains instructions for how the SolrDispatchFilter
+       should behave when processing requests for this SolrCore.
+
+       handleSelect affects the behavior of requests such as /select?qt=XXX
+
+       handleSelect="true" will cause the SolrDispatchFilter to process
+       the request and will result in consistent error handling and
+       formatting for all types of requests.
+
+       handleSelect="false" will cause the SolrDispatchFilter to
+       ignore "/select" requests and fallback to using the legacy
+       SolrServlet and it's Solr 1.1 style error formatting
+    -->
+  <requestDispatcher handleSelect="true" >
+    <!-- Request Parsing
+
+         These settings indicate how Solr Requests may be parsed, and
+         what restrictions may be placed on the ContentStreams from
+         those requests
+
+         enableRemoteStreaming - enables use of the stream.file
+         and stream.url parameters for specifying remote streams.
+
+         multipartUploadLimitInKB - specifies the max size of
+         Multipart File Uploads that Solr will allow in a Request.
+         
+         *** WARNING ***
+         The settings below authorize Solr to fetch remote files, You
+         should make sure your system has some authentication before
+         using enableRemoteStreaming="true"
+
+      --> 
+    <requestParsers enableRemoteStreaming="true" 
+                    multipartUploadLimitInKB="2048000" />
+
+    <!-- HTTP Caching
+
+         Set HTTP caching related parameters (for proxy caches and clients).
+
+         The options below instruct Solr not to output any HTTP Caching
+         related headers
+      -->
+    <httpCaching never304="true" />
+    <!-- If you include a <cacheControl> directive, it will be used to
+         generate a Cache-Control header (as well as an Expires header
+         if the value contains "max-age=")
+         
+         By default, no Cache-Control header is generated.
+         
+         You can use the <cacheControl> option even if you have set
+         never304="true"
+      -->
+    <!--
+       <httpCaching never304="true" >
+         <cacheControl>max-age=30, public</cacheControl> 
+       </httpCaching>
+      -->
+    <!-- To enable Solr to respond with automatically generated HTTP
+         Caching headers, and to response to Cache Validation requests
+         correctly, set the value of never304="false"
+         
+         This will cause Solr to generate Last-Modified and ETag
+         headers based on the properties of the Index.
+
+         The following options can also be specified to affect the
+         values of these headers...
+
+         lastModFrom - the default value is "openTime" which means the
+         Last-Modified value (and validation against If-Modified-Since
+         requests) will all be relative to when the current Searcher
+         was opened.  You can change it to lastModFrom="dirLastMod" if
+         you want the value to exactly correspond to when the physical
+         index was last modified.
+
+         etagSeed="..." is an option you can change to force the ETag
+         header (and validation against If-None-Match requests) to be
+         different even if the index has not changed (ie: when making
+         significant changes to your config file)
+
+         (lastModifiedFrom and etagSeed are both ignored if you use
+         the never304="true" option)
+      -->
+    <!--
+       <httpCaching lastModifiedFrom="openTime"
+                    etagSeed="Solr">
+         <cacheControl>max-age=30, public</cacheControl> 
+       </httpCaching>
+      -->
+  </requestDispatcher>
+
+  <!-- Request Handlers 
+
+       http://wiki.apache.org/solr/SolrRequestHandler
+
+       incoming queries will be dispatched to the correct handler
+       based on the path or the qt (query type) param.
+
+       Names starting with a '/' are accessed with the a path equal to
+       the registered name.  Names without a leading '/' are accessed
+       with: http://host/app/[core/]select?qt=name
+
+       If a /select request is processed with out a qt param
+       specified, the requestHandler that declares default="true" will
+       be used.
+       
+       If a Request Handler is declared with startup="lazy", then it will
+       not be initialized until the first request that uses it.
+
+    -->
+  <!-- SearchHandler
+
+       http://wiki.apache.org/solr/SearchHandler
+
+       For processing Search Queries, the primary Request Handler
+       provided with Solr is "SearchHandler" It delegates to a sequent
+       of SearchComponents (see below) and supports distributed
+       queries across multiple shards
+    -->
+  <requestHandler name="search" class="solr.SearchHandler" default="true">
+    <!-- default values for query parameters can be specified, these
+         will be overridden by parameters in the request
+      -->
+     <lst name="defaults">
+       <str name="echoParams">explicit</str>
+       <int name="rows">10</int>
+     </lst>
+    <!-- In addition to defaults, "appends" params can be specified
+         to identify values which should be appended to the list of
+         multi-val params from the query (or the existing "defaults").
+      -->
+    <!-- In this example, the param "fq=instock:true" would be appended to
+         any query time fq params the user may specify, as a mechanism for
+         partitioning the index, independent of any user selected filtering
+         that may also be desired (perhaps as a result of faceted searching).
+
+         NOTE: there is *absolutely* nothing a client can do to prevent these
+         "appends" values from being used, so don't use this mechanism
+         unless you are sure you always want it.
+      -->
+    <!--
+       <lst name="appends">
+         <str name="fq">inStock:true</str>
+       </lst>
+      -->
+    <!-- "invariants" are a way of letting the Solr maintainer lock down
+         the options available to Solr clients.  Any params values
+         specified here are used regardless of what values may be specified
+         in either the query, the "defaults", or the "appends" params.
+
+         In this example, the facet.field and facet.query params would
+         be fixed, limiting the facets clients can use.  Faceting is
+         not turned on by default - but if the client does specify
+         facet=true in the request, these are the only facets they
+         will be able to see counts for; regardless of what other
+         facet.field or facet.query params they may specify.
+
+         NOTE: there is *absolutely* nothing a client can do to prevent these
+         "invariants" values from being used, so don't use this mechanism
+         unless you are sure you always want it.
+      -->
+    <!--
+       <lst name="invariants">
+         <str name="facet.field">cat</str>
+         <str name="facet.field">manu_exact</str>
+         <str name="facet.query">price:[* TO 500]</str>
+         <str name="facet.query">price:[500 TO *]</str>
+       </lst>
+      -->
+    <!-- If the default list of SearchComponents is not desired, that
+         list can either be overridden completely, or components can be
+         prepended or appended to the default list.  (see below)
+      -->
+    <!--
+       <arr name="components">
+         <str>nameOfCustomComponent1</str>
+         <str>nameOfCustomComponent2</str>
+       </arr>
+      -->
+    </requestHandler>
+
+  <!-- A Robust Example
+
+       This example SearchHandler declaration shows off usage of the
+       SearchHandler with many defaults declared
+
+       Note that multiple instances of the same Request Handler
+       (SearchHandler) can be registered multiple times with different
+       names (and different init parameters)
+    -->
+  <requestHandler name="/browse" class="solr.SearchHandler">
+     <lst name="defaults">
+       <str name="echoParams">explicit</str>
+
+       <!-- VelocityResponseWriter settings -->
+       <str name="wt">velocity</str>
+
+       <str name="v.template">browse</str>
+       <str name="v.layout">layout</str>
+       <str name="title">Solritas</str>
+
+       <str name="defType">edismax</str>
+       <str name="q.alt">*:*</str>
+       <str name="rows">10</str>
+       <str name="fl">*,score</str>
+       <str name="mlt.qf">
+         text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
+       </str>
+       <str name="mlt.fl">text,features,name,sku,id,manu,cat</str>
+       <int name="mlt.count">3</int>
+
+       <str name="qf">
+          text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
+       </str>
+
+       <str name="facet">on</str>
+       <str name="facet.field">cat</str>
+       <str name="facet.field">manu_exact</str>
+       <str name="facet.query">ipod</str>
+       <str name="facet.query">GB</str>
+       <str name="facet.mincount">1</str>
+       <str name="facet.pivot">cat,inStock</str>
+       <str name="facet.range">price</str>
+       <int name="f.price.facet.range.start">0</int>
+       <int name="f.price.facet.range.end">600</int>
+       <int name="f.price.facet.range.gap">50</int>
+       <str name="f.price.facet.range.other">after</str>
+       <str name="facet.range">manufacturedate_dt</str>
+       <str name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
+       <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
+       <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
+       <str name="f.manufacturedate_dt.facet.range.other">before</str>
+       <str name="f.manufacturedate_dt.facet.range.other">after</str>
+
+
+       <!-- Highlighting defaults -->
+       <str name="hl">on</str>
+       <str name="hl.fl">text features name</str>
+       <str name="f.name.hl.fragsize">0</str>
+       <str name="f.name.hl.alternateField">name</str>
+     </lst>
+     <arr name="last-components">
+       <str>spellcheck</str>
+     </arr>
+     <!--
+     <str name="url-scheme">httpx</str>
+     -->
+  </requestHandler>
+
+  <!-- XML Update Request Handler.  
+       
+       http://wiki.apache.org/solr/UpdateXmlMessages
+
+       The canonical Request Handler for Modifying the Index through
+       commands specified using XML.
+
+       Note: Since solr1.1 requestHandlers requires a valid content
+       type header if posted in the body. For example, curl now
+       requires: -H 'Content-type:text/xml; charset=utf-8'
+    -->
+  <requestHandler name="/update" 
+                  class="solr.XmlUpdateRequestHandler">
+    <!-- See below for information on defining 
+         updateRequestProcessorChains that can be used by name 
+         on each Update Request
+      -->
+    <!--
+       <lst name="defaults">
+         <str name="update.chain">dedupe</str>
+       </lst>
+       -->
+    </requestHandler>
+  <!-- Binary Update Request Handler
+       http://wiki.apache.org/solr/javabin
+    -->
+  <requestHandler name="/update/javabin" 
+                  class="solr.BinaryUpdateRequestHandler" />
+
+  <!-- CSV Update Request Handler
+       http://wiki.apache.org/solr/UpdateCSV
+    -->
+  <requestHandler name="/update/csv" 
+                  class="solr.CSVRequestHandler" 
+                  startup="lazy" />
+
+  <!-- JSON Update Request Handler
+       http://wiki.apache.org/solr/UpdateJSON
+    -->
+  <requestHandler name="/update/json" 
+                  class="solr.JsonUpdateRequestHandler" 
+                  startup="lazy" />
+
+  <!-- Solr Cell Update Request Handler
+
+       http://wiki.apache.org/solr/ExtractingRequestHandler 
+
+    -->
+  <requestHandler name="/update/extract" 
+                  startup="lazy"
+                  class="solr.extraction.ExtractingRequestHandler" >
+    <lst name="defaults">
+      <!-- All the main content goes into "text"... if you need to return
+           the extracted text or do highlighting, use a stored field. -->
+      <str name="fmap.content">text</str>
+      <str name="lowernames">true</str>
+      <str name="uprefix">ignored_</str>
+
+      <!-- capture link hrefs but ignore div attributes -->
+      <str name="captureAttr">true</str>
+      <str name="fmap.a">links</str>
+      <str name="fmap.div">ignored_</str>
+    </lst>
+  </requestHandler>
+
+  <!-- Field Analysis Request Handler
+
+       RequestHandler that provides much the same functionality as
+       analysis.jsp. Provides the ability to specify multiple field
+       types and field names in the same request and outputs
+       index-time and query-time analysis for each of them.
+
+       Request parameters are:
+       analysis.fieldname - field name whose analyzers are to be used
+
+       analysis.fieldtype - field type whose analyzers are to be used
+       analysis.fieldvalue - text for index-time analysis
+       q (or analysis.q) - text for query time analysis
+       analysis.showmatch (true|false) - When set to true and when
+           query analysis is performed, the produced tokens of the
+           field value analysis will be marked as "matched" for every
+           token that is produces by the query analysis
+   -->
+  <requestHandler name="/analysis/field" 
+                  startup="lazy"
+                  class="solr.FieldAnalysisRequestHandler" />
+
+
+  <!-- Document Analysis Handler
+
+       http://wiki.apache.org/solr/AnalysisRequestHandler
+
+       An analysis handler that provides a breakdown of the analysis
+       process of provided docuemnts. This handler expects a (single)
+       content stream with the following format:
+
+       <docs>
+         <doc>
+           <field name="id">1</field>
+           <field name="name">The Name</field>
+           <field name="text">The Text Value</field>
+         </doc>
+         <doc>...</doc>
+         <doc>...</doc>
+         ...
+       </docs>
+
+    Note: Each document must contain a field which serves as the
+    unique key. This key is used in the returned response to associate
+    an analysis breakdown to the analyzed document.
+
+    Like the FieldAnalysisRequestHandler, this handler also supports
+    query analysis by sending either an "analysis.query" or "q"
+    request parameter that holds the query text to be analyzed. It
+    also supports the "analysis.showmatch" parameter which when set to
+    true, all field tokens that match the query tokens will be marked
+    as a "match". 
+  -->
+  <requestHandler name="/analysis/document" 
+                  class="solr.DocumentAnalysisRequestHandler" 
+                  startup="lazy" />
+
+  <!-- Admin Handlers
+
+       Admin Handlers - This will register all the standard admin
+       RequestHandlers.  
+    -->
+  <requestHandler name="/admin/" 
+                  class="solr.admin.AdminHandlers" />
+  <!-- This single handler is equivalent to the following... -->
+  <!--
+     <requestHandler name="/admin/luke"       class="solr.admin.LukeRequestHandler" />
+     <requestHandler name="/admin/system"     class="solr.admin.SystemInfoHandler" />
+     <requestHandler name="/admin/plugins"    class="solr.admin.PluginInfoHandler" />
+     <requestHandler name="/admin/threads"    class="solr.admin.ThreadDumpHandler" />
+     <requestHandler name="/admin/properties" class="solr.admin.PropertiesRequestHandler" />
+     <requestHandler name="/admin/file"       class="solr.admin.ShowFileRequestHandler" >
+    -->
+  <!-- If you wish to hide files under ${solr.home}/conf, explicitly
+       register the ShowFileRequestHandler using: 
+    -->
+  <!--
+     <requestHandler name="/admin/file" 
+                     class="solr.admin.ShowFileRequestHandler" >
+       <lst name="invariants">
+         <str name="hidden">synonyms.txt</str> 
+         <str name="hidden">anotherfile.txt</str> 
+       </lst>
+     </requestHandler>
+    -->
+
+  <!-- ping/healthcheck -->
+  <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
+    <lst name="defaults">
+      <str name="qt">search</str>
+      <str name="q">solrpingquery</str>
+      <str name="echoParams">all</str>
+    </lst>
+  </requestHandler>
+
+  <!-- Echo the request contents back to the client -->
+  <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
+    <lst name="defaults">
+     <str name="echoParams">explicit</str> 
+     <str name="echoHandler">true</str>
+    </lst>
+  </requestHandler>
+  
+  <!-- Solr Replication
+
+       The SolrReplicationHandler supports replicating indexes from a
+       "master" used for indexing and "salves" used for queries.
+
+       http://wiki.apache.org/solr/SolrReplication 
+
+       In the example below, remove the <lst name="master"> section if
+       this is just a slave and remove  the <lst name="slave"> section
+       if this is just a master.
+    -->
+  <!--
+     <requestHandler name="/replication" class="solr.ReplicationHandler" >
+       <lst name="master">
+         <str name="replicateAfter">commit</str>
+         <str name="replicateAfter">startup</str>
+         <str name="confFiles">schema.xml,stopwords.txt</str>
+       </lst>
+       <lst name="slave">
+         <str name="masterUrl">http://localhost:8983/solr/replication</str>
+         <str name="pollInterval">00:00:60</str>
+       </lst>
+     </requestHandler>
+    -->
+
+  <!-- Search Components
+
+       Search components are registered to SolrCore and used by 
+       instances of SearchHandler (which can access them by name)
+       
+       By default, the following components are available:
+       
+       <searchComponent name="query"     class="solr.QueryComponent" />
+       <searchComponent name="facet"     class="solr.FacetComponent" />
+       <searchComponent name="mlt"       class="solr.MoreLikeThisComponent" />
+       <searchComponent name="highlight" class="solr.HighlightComponent" />
+       <searchComponent name="stats"     class="solr.StatsComponent" />
+       <searchComponent name="debug"     class="solr.DebugComponent" />
+   
+       Default configuration in a requestHandler would look like:
+
+       <arr name="components">
+         <str>query</str>
+         <str>facet</str>
+         <str>mlt</str>
+         <str>highlight</str>
+         <str>stats</str>
+         <str>debug</str>
+       </arr>
+
+       If you register a searchComponent to one of the standard names, 
+       that will be used instead of the default.
+
+       To insert components before or after the 'standard' components, use:
+    
+       <arr name="first-components">
+         <str>myFirstComponentName</str>
+       </arr>
+    
+       <arr name="last-components">
+         <str>myLastComponentName</str>
+       </arr>
+
+       NOTE: The component registered with the name "debug" will
+       always be executed after the "last-components" 
+       
+     -->
+
+   <!-- Spell Check
+
+        The spell check component can return a list of alternative spelling
+        suggestions.  
+
+        http://wiki.apache.org/solr/SpellCheckComponent
+     -->
+  <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
+
+    <str name="queryAnalyzerFieldType">textSpell</str>
+
+    <!-- Multiple "Spell Checkers" can be declared and used by this
+         component
+      -->
+
+    <!-- a spellchecker built from a field of the main index, and
+         written to disk
+      -->
+    <lst name="spellchecker">
+      <str name="name">default</str>
+      <str name="field">name</str>
+      <str name="spellcheckIndexDir">spellchecker</str>
+      <!-- uncomment this to require terms to occur in 1% of the documents in order to be included in the dictionary
+      	<float name="thresholdTokenFrequency">.01</float>
+      -->
+    </lst>
+
+    <!-- a spellchecker that uses a different distance measure -->
+    <!--
+       <lst name="spellchecker">
+         <str name="name">jarowinkler</str>
+         <str name="field">spell</str>
+         <str name="distanceMeasure">
+           org.apache.lucene.search.spell.JaroWinklerDistance
+         </str>
+         <str name="spellcheckIndexDir">spellcheckerJaro</str>
+       </lst>
+     -->
+
+    <!-- a spellchecker that use an alternate comparator 
+
+         comparatorClass be one of:
+          1. score (default)
+          2. freq (Frequency first, then score)
+          3. A fully qualified class name
+      -->
+    <!--
+       <lst name="spellchecker">
+         <str name="name">freq</str>
+         <str name="field">lowerfilt</str>
+         <str name="spellcheckIndexDir">spellcheckerFreq</str>
+         <str name="comparatorClass">freq</str>
+         <str name="buildOnCommit">true</str>
+      -->
+
+    <!-- A spellchecker that reads the list of words from a file -->
+    <!--
+       <lst name="spellchecker">
+         <str name="classname">solr.FileBasedSpellChecker</str>
+         <str name="name">file</str>
+         <str name="sourceLocation">spellings.txt</str>
+         <str name="characterEncoding">UTF-8</str>
+         <str name="spellcheckIndexDir">spellcheckerFile</str>
+       </lst>
+      -->
+  </searchComponent>
+
+  <!-- A request handler for demonstrating the spellcheck component.  
+
+       NOTE: This is purely as an example.  The whole purpose of the
+       SpellCheckComponent is to hook it into the request handler that
+       handles your normal user queries so that a separate request is
+       not needed to get suggestions.
+
+       IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS
+       NOT WHAT YOU WANT FOR YOUR PRODUCTION SYSTEM!
+       
+       See http://wiki.apache.org/solr/SpellCheckComponent for details
+       on the request parameters.
+    -->
+  <requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
+    <lst name="defaults">
+      <str name="spellcheck.onlyMorePopular">false</str>
+      <str name="spellcheck.extendedResults">false</str>
+      <str name="spellcheck.count">1</str>
+    </lst>
+    <arr name="last-components">
+      <str>spellcheck</str>
+    </arr>
+  </requestHandler>
+
+  <!-- Term Vector Component
+
+       http://wiki.apache.org/solr/TermVectorComponent
+    -->
+  <searchComponent name="tvComponent" class="solr.TermVectorComponent"/>
+
+  <!-- A request handler for demonstrating the term vector component
+
+       This is purely as an example.
+
+       In reality you will likely want to add the component to your 
+       already specified request handlers. 
+    -->
+  <requestHandler name="tvrh" class="solr.SearchHandler" startup="lazy">
+    <lst name="defaults">
+      <bool name="tv">true</bool>
+    </lst>
+    <arr name="last-components">
+      <str>tvComponent</str>
+    </arr>
+  </requestHandler>
+
+  <!-- Clustering Component
+
+       http://wiki.apache.org/solr/ClusteringComponent
+
+       This relies on third party jars which are notincluded in the
+       release.  To use this component (and the "/clustering" handler)
+       Those jars will need to be downloaded, and you'll need to set
+       the solr.cluster.enabled system property when running solr...
+
+          java -Dsolr.clustering.enabled=true -jar start.jar
+    -->
+  <searchComponent name="clustering" 
+                   enable="${solr.clustering.enabled:false}"
+                   class="solr.clustering.ClusteringComponent" >
+    <!-- Declare an engine -->
+    <lst name="engine">
+      <!-- The name, only one can be named "default" -->
+      <str name="name">default</str>
+
+      <!-- Class name of Carrot2 clustering algorithm. 
+           
+           Currently available algorithms are:
+           
+           * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
+           * org.carrot2.clustering.stc.STCClusteringAlgorithm
+           * org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm
+           
+           See http://project.carrot2.org/algorithms.html for the
+           algorithm's characteristics.
+        -->
+      <str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
+
+      <!-- Overriding values for Carrot2 default algorithm attributes.
+
+           For a description of all available attributes, see:
+           http://download.carrot2.org/stable/manual/#chapter.components.
+           Use attribute key as name attribute of str elements
+           below. These can be further overridden for individual
+           requests by specifying attribute key as request parameter
+           name and attribute value as parameter value.
+        -->
+      <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
+      
+      <!-- Location of Carrot2 lexical resources.
+
+           A directory from which to load Carrot2-specific stop words
+           and stop labels. Absolute or relative to Solr config directory.
+           If a specific resource (e.g. stopwords.en) is present in the
+           specified dir, it will completely override the corresponding
+           default one that ships with Carrot2.
+
+           For an overview of Carrot2 lexical resources, see:
+           http://download.carrot2.org/head/manual/#chapter.lexical-resources
+        -->
+      <str name="carrot.lexicalResourcesDir">clustering/carrot2</str>
+
+      <!-- The language to assume for the documents.
+           
+           For a list of allowed values, see:
+           http://download.carrot2.org/stable/manual/#section.attribute.lingo.MultilingualClustering.defaultLanguage
+       -->
+      <str name="MultilingualClustering.defaultLanguage">ENGLISH</str>
+    </lst>
+    <lst name="engine">
+      <str name="name">stc</str>
+      <str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
+    </lst>
+  </searchComponent>
+
+  <!-- A request handler for demonstrating the clustering component
+
+       This is purely as an example.
+
+       In reality you will likely want to add the component to your 
+       already specified request handlers. 
+    -->
+  <requestHandler name="/clustering"
+                  startup="lazy"
+                  enable="${solr.clustering.enabled:false}"
+                  class="solr.SearchHandler">
+    <lst name="defaults">
+      <bool name="clustering">true</bool>
+      <str name="clustering.engine">default</str>
+      <bool name="clustering.results">true</bool>
+      <!-- The title field -->
+      <str name="carrot.title">name</str>
+      <str name="carrot.url">id</str>
+      <!-- The field to cluster on -->
+       <str name="carrot.snippet">features</str>
+       <!-- produce summaries -->
+       <bool name="carrot.produceSummary">true</bool>
+       <!-- the maximum number of labels per cluster -->
+       <!--<int name="carrot.numDescriptions">5</int>-->
+       <!-- produce sub clusters -->
+       <bool name="carrot.outputSubClusters">false</bool>
+       
+       <str name="defType">edismax</str>
+       <str name="qf">
+          text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
+       </str>
+       <str name="q.alt">*:*</str>
+       <str name="rows">10</str>
+       <str name="fl">*,score</str>
+    </lst>     
+    <arr name="last-components">
+      <str>clustering</str>
+    </arr>
+  </requestHandler>
+  
+  <!-- Terms Component
+
+       http://wiki.apache.org/solr/TermsComponent
+
+       A component to return terms and document frequency of those
+       terms
+    -->
+  <searchComponent name="terms" class="solr.TermsComponent"/>
+
+  <!-- A request handler for demonstrating the terms component -->
+  <requestHandler name="/terms" class="solr.SearchHandler" startup="lazy">
+     <lst name="defaults">
+      <bool name="terms">true</bool>
+    </lst>     
+    <arr name="components">
+      <str>terms</str>
+    </arr>
+  </requestHandler>
+
+
+  <!-- Query Elevation Component
+
+       http://wiki.apache.org/solr/QueryElevationComponent
+
+       a search component that enables you to configure the top
+       results for a given query regardless of the normal lucene
+       scoring.
+    -->
+  <searchComponent name="elevator" class="solr.QueryElevationComponent" >
+    <!-- pick a fieldType to analyze queries -->
+    <str name="queryFieldType">string</str>
+    <str name="config-file">elevate.xml</str>
+  </searchComponent>
+
+  <!-- A request handler for demonstrating the elevator component -->
+  <requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
+    <lst name="defaults">
+      <str name="echoParams">explicit</str>
+    </lst>
+    <arr name="last-components">
+      <str>elevator</str>
+    </arr>
+  </requestHandler>
+
+  <!-- Highlighting Component
+
+       http://wiki.apache.org/solr/HighlightingParameters
+    -->
+  <searchComponent class="solr.HighlightComponent" name="highlight">
+    <highlighting>
+      <!-- Configure the standard fragmenter -->
+      <!-- This could most likely be commented out in the "default" case -->
+      <fragmenter name="gap" 
+                  default="true"
+                  class="solr.highlight.GapFragmenter">
+        <lst name="defaults">
+          <int name="hl.fragsize">100</int>
+        </lst>
+      </fragmenter>
+
+      <!-- A regular-expression-based fragmenter 
+           (for sentence extraction) 
+        -->
+      <fragmenter name="regex" 
+                  class="solr.highlight.RegexFragmenter">
+        <lst name="defaults">
+          <!-- slightly smaller fragsizes work better because of slop -->
+          <int name="hl.fragsize">70</int>
+          <!-- allow 50% slop on fragment sizes -->
+          <float name="hl.regex.slop">0.5</float>
+          <!-- a basic sentence pattern -->
+          <str name="hl.regex.pattern">[-\w ,/\n\&quot;&apos;]{20,200}</str>
+        </lst>
+      </fragmenter>
+
+      <!-- Configure the standard formatter -->
+      <formatter name="html" 
+                 default="true"
+                 class="solr.highlight.HtmlFormatter">
+        <lst name="defaults">
+          <str name="hl.simple.pre"><![CDATA[<em>]]></str>
+          <str name="hl.simple.post"><![CDATA[</em>]]></str>
+        </lst>
+      </formatter>
+
+      <!-- Configure the standard encoder -->
+      <encoder name="html" 
+               class="solr.highlight.HtmlEncoder" />
+
+      <!-- Configure the standard fragListBuilder -->
+      <fragListBuilder name="simple" 
+                       default="true"
+                       class="solr.highlight.SimpleFragListBuilder"/>
+
+      <!-- Configure the single fragListBuilder -->
+      <fragListBuilder name="single" 
+                       class="solr.highlight.SingleFragListBuilder"/>
+
+      <!-- default tag FragmentsBuilder -->
+      <fragmentsBuilder name="default" 
+                        default="true"
+                        class="solr.highlight.ScoreOrderFragmentsBuilder">
+        <!-- 
+        <lst name="defaults">
+          <str name="hl.multiValuedSeparatorChar">/</str>
+        </lst>
+        -->
+      </fragmentsBuilder>
+
+      <!-- multi-colored tag FragmentsBuilder -->
+      <fragmentsBuilder name="colored" 
+                        class="solr.highlight.ScoreOrderFragmentsBuilder">
+        <lst name="defaults">
+          <str name="hl.tag.pre"><![CDATA[
+               <b style="background:yellow">,<b style="background:lawgreen">,
+               <b style="background:aquamarine">,<b style="background:magenta">,
+               <b style="background:palegreen">,<b style="background:coral">,
+               <b style="background:wheat">,<b style="background:khaki">,
+               <b style="background:lime">,<b style="background:deepskyblue">]]></str>
+          <str name="hl.tag.post"><![CDATA[</b>]]></str>
+        </lst>
+      </fragmentsBuilder>
+    </highlighting>
+  </searchComponent>
+
+  <!-- Update Processors
+
+       Chains of Update Processor Factories for dealing with Update
+       Requests can be declared, and then used by name in Update
+       Request Processors
+
+       http://wiki.apache.org/solr/UpdateRequestProcessor
+
+    --> 
+  <!-- Deduplication
+
+       An example dedup update processor that creates the "id" field
+       on the fly based on the hash code of some other fields.  This
+       example has overwriteDupes set to false since we are using the
+       id field as the signatureField and Solr will maintain
+       uniqueness based on that anyway.  
+       
+    -->
+  <!--
+     <updateRequestProcessorChain name="dedupe">
+       <processor class="solr.processor.SignatureUpdateProcessorFactory">
+         <bool name="enabled">true</bool>
+         <str name="signatureField">id</str>
+         <bool name="overwriteDupes">false</bool>
+         <str name="fields">name,features,cat</str>
+         <str name="signatureClass">solr.processor.Lookup3Signature</str>
+       </processor>
+       <processor class="solr.LogUpdateProcessorFactory" />
+       <processor class="solr.RunUpdateProcessorFactory" />
+     </updateRequestProcessorChain>
+    -->
+
+  <!-- Response Writers
+
+       http://wiki.apache.org/solr/QueryResponseWriter
+
+       Request responses will be written using the writer specified by
+       the 'wt' request parameter matching the name of a registered
+       writer.
+
+       The "default" writer is the default and will be used if 'wt' is
+       not specified in the request.
+    -->
+  <!-- The following response writers are implicitly configured unless
+       overridden...
+    -->
+  <!--
+     <queryResponseWriter name="xml" 
+                          default="true"
+                          class="solr.XMLResponseWriter" />
+     <queryResponseWriter name="json" class="solr.JSONResponseWriter"/>
+     <queryResponseWriter name="python" class="solr.PythonResponseWriter"/>
+     <queryResponseWriter name="ruby" class="solr.RubyResponseWriter"/>
+     <queryResponseWriter name="php" class="solr.PHPResponseWriter"/>
+     <queryResponseWriter name="phps" class="solr.PHPSerializedResponseWriter"/>
+     <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter"/>
+     <queryResponseWriter name="csv" class="solr.CSVResponseWriter"/>
+    -->
+  <!--
+     Custom response writers can be declared as needed...
+    -->
+  <!--
+     <queryResponseWriter name="custom" class="com.example.MyResponseWriter"/>
+    -->
+
+  <!-- XSLT response writer transforms the XML output by any xslt file found
+       in Solr's conf/xslt directory.  Changes to xslt files are checked for
+       every xsltCacheLifetimeSeconds.  
+    -->
+  <queryResponseWriter name="xslt" class="solr.XSLTResponseWriter">
+    <int name="xsltCacheLifetimeSeconds">5</int>
+  </queryResponseWriter>
+
+  <!-- Query Parsers
+
+       http://wiki.apache.org/solr/SolrQuerySyntax
+
+       Multiple QParserPlugins can be registered by name, and then
+       used in either the "defType" param for the QueryComponent (used
+       by SearchHandler) or in LocalParams
+    -->
+  <!-- example of registering a query parser -->
+  <!--
+     <queryParser name="myparser" class="com.mycompany.MyQParserPlugin"/>
+    -->
+
+  <!-- Function Parsers
+
+       http://wiki.apache.org/solr/FunctionQuery
+
+       Multiple ValueSourceParsers can be registered by name, and then
+       used as function names when using the "func" QParser.
+    -->
+  <!-- example of registering a custom function parser  -->
+  <!--
+     <valueSourceParser name="myfunc" 
+                        class="com.mycompany.MyValueSourceParser" />
+    -->
+
+  <!-- Legacy config for the admin interface -->
+  <admin>
+    <defaultQuery>*:*</defaultQuery>
+
+    <!-- configure a healthcheck file for servers behind a
+         loadbalancer 
+      -->
+    <!--
+       <healthcheck type="file">server-enabled</healthcheck>
+      -->
+  </admin>
+
+  <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
+    <lst name="defaults">
+      <str name="config">/etc/solr/data-config.xml</str>
+    </lst>
+  </requestHandler>
+
+
+</config>
@@ -1,16 +0,0 @@
-Installation instructions for Ubuntu 10.04
--------------------------------------------
-
-* Install Java, tomcat, Solr and JDBC Postgres drivers (Ubuntu partner repositories must be enabled):
-sudo apt-get install sun-java6-bin tomcat6 solr-common libpg-java
-
-* Link the JDBC Postgres drivers into the Solr installation:
-sudo ln -s /usr/share/java/postgresql-jdbc3-8.4.jar /usr/share/solr/WEB-INF/lib/
-
-* Check data-config.xml to make sure all information to connect to the databases are right
-
-* Copy the configuration files from this folder into /etc/solr/conf/
-
-* Restart tomcat:
-sudo /etc/init.d/tomcat6 restart
-
@@ -1,314 +0,0 @@
- <dataConfig>
-  <dataSource name="trac"
-              type="JdbcDataSource" 
-              driver="org.postgresql.Driver"
-              url="jdbc:postgresql://localhost/trac_gitec" 
-              user="colab" />
-  <dataSource name="colab"
-              type="JdbcDataSource" 
-              driver="org.postgresql.Driver"
-              url="jdbc:postgresql://localhost/colab" 
-              user="colab" />
-
-  <document>
-
-    <entity name="wiki"
-            dataSource="trac"
-            transformer="TemplateTransformer,DateFormatTransformer"
-            query="SELECT
-                     name,
-                     TIMESTAMP 'epoch' + max(time) * INTERVAL '1s' AS modified,
-                     max(version) AS version
-                   FROM wiki GROUP BY name"
-            deltaQuery="
-              SELECT DISTINCT
-                name
-              FROM
-                wiki
-              WHERE
-                time > EXTRACT(
-                  epoch FROM TIMESTAMP '${dataimporter.wiki.last_index_time}'
-                )"
-            deltaImportQuery="
-                   SELECT
-                     name,
-                     TIMESTAMP 'epoch' + max(time) * INTERVAL '1s' AS modified,
-                     max(version) AS version
-                   FROM 
-                     wiki
-                   WHERE
-                     name = '${dataimporter.delta.id}'
-                   GROUP BY name">
-      
-      <entity name="wiki_creation" 
-              dataSource="trac"
-              query="SELECT 
-                       author AS creator,
-                       TIMESTAMP 'epoch' + time * INTERVAL '1s' AS created
-                     FROM 
-                       wiki 
-                     WHERE
-                       name = '${wiki.name}' 
-                       AND version = 1" />
-      
-      <entity name="wiki_collaborators"
-              dataSource="trac"
-              query="SELECT DISTINCT 
-                       author AS collaborator
-                     FROM 
-                       wiki 
-                     WHERE
-                       name = '${wiki.name}'
-                       AND author != ''" />
-      
-      <entity name="content"
-              dataSource="trac"
-              query="SELECT 
-                       text AS content 
-                     FROM
-                       wiki 
-                     WHERE 
-                       name = '${wiki.name}'
-                       AND version = '${wiki.version}'" />
-
-      <field column="uid" template="WIKI_${wiki.name}" />
-      <field column="id" template="${wiki.name}" />
-      <field column="type" template="wiki" />
-      <field column="title" template="${wiki.name}" />
-      <field column="created" name="created" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
-      <field column="modified" name="modified" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
-    </entity>
-
-    <entity name="ticket"
-            dataSource="trac"
-            transformer="TemplateTransformer,DateFormatTransformer"
-            pk="id"
-            deltaQuery="
-              SELECT
-                  id
-                FROM
-                  ticket
-              WHERE
-                time > EXTRACT(
-                  epoch FROM TIMESTAMP '${dataimporter.ticket.last_index_time}'
-                )"
-            query="SELECT
-                     id, 
-                     summary,
-                     description,
-                     milestone,
-                     priority,
-                     component,
-                     version,
-                     severity,
-                     reporter,
-                     owner,
-                     status,
-                     TIMESTAMP 'epoch' + time * INTERVAL '1s' AS created,
-                     TIMESTAMP 'epoch' + changetime * INTERVAL '1s' AS modified
-                   FROM
-                     ticket">
-
-      <entity name="ticket_collaborator"
-              dataSource="trac"
-              query="SELECT 
-                       reporter AS collaborator 
-                     FROM
-                       ticket
-                     WHERE 
-                       id = ${ticket.id}
-                     
-                     UNION
-                     
-                     SELECT 
-                       owner AS collaborator 
-                     FROM
-                       ticket
-                     WHERE 
-                       id = ${ticket.id}
-                     
-                     UNION
-                    
-                     SELECT DISTINCT
-                       author AS collaborator
-                     FROM 
-                       ticket_change 
-                     WHERE
-                       ticket = ${ticket.id}" />
-
-      <entity name="ticket_keywords"
-              dataSource="trac"
-              query="SELECT DISTINCT
-                       REGEXP_SPLIT_TO_TABLE(keywords, ',|\\s') AS keyword
-                     FROM 
-                       ticket
-                     WHERE
-                       id = ${ticket.id} AND
-                       keywords != ''" />
-      
-      <entity name="ticket_comments"
-              dataSource="trac"
-              query="SELECT
-                       newvalue AS comment
-                     FROM 
-                       ticket_change
-                     WHERE
-                       ticket = ${ticket.id}
-                       AND field = 'comment'" />
-
-      <field column="uid" template="TICKET_${ticket.id}" />
-      <field column="type" template="ticket" />
-      <field column="title" 
-             template="#${ticket.id} (${ticket.status}) - ${ticket.summary}" />
-      <field column="creator" template="${ticket.reporter}" />
-      <field column="created" name="created" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
-      <field column="modified" name="modified" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
-    </entity>
-    
-    <entity name="changeset"
-            dataSource="trac"
-            transformer="TemplateTransformer,DateFormatTransformer"
-            pk="rev"
-            deltaQuery="
-            SELECT 
-              rev 
-            FROM 
-              revision 
-            WHERE
-              time > EXTRACT(
-                epoch FROM TIMESTAMP '${dataimporter.changeset.last_index_time}'
-              )"
-
-            query="SELECT
-                     rev AS revision, 
-                     author AS creator,
-                     author AS collaborator,
-                     TIMESTAMP 'epoch' + time * INTERVAL '1s' AS created,
-                     message
-                   FROM
-                     revision">
-      <field column="uid" template="CHANGESET_${changeset.revision}" />
-      <field column="id" template="${changeset.revision}" />
-      <field column="type" template="changeset" />
-      <field column="title" 
-             template="[${changeset.revision}] - ${changeset.message}" />
-      <field column="created" name="created" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss"/>
-    </entity>
-
-    <entity name="thread"
-            dataSource="colab"
-            transformer="TemplateTransformer,DateFormatTransformer"
-            deltaQuery="
-              SELECT
-                thread_id AS id
-              FROM
-                super_archives_message
-              GROUP BY
-                thread_id
-              HAVING
-                max(received_time) > '${dataimporter.thread.last_index_time}'"
-            deltaImportQuery="SELECT
-                     sam.thread_id AS id,
-                     sat.subject_token AS name,
-                     sat.latest_message_id,
-                     saml.name AS mailinglist,
-                     array_to_string(array_agg(sam.body), ' ') AS content
-                   FROM
-                     super_archives_message AS sam
-                     JOIN super_archives_thread AS sat
-                       ON sat.id = sam.thread_id
-                     JOIN super_archives_mailinglist AS saml
-                       ON sat.mailinglist_id = saml.id
-                   WHERE
-                     sat.id = '${dataimporter.delta.id}'
-                   GROUP BY
-                     sam.thread_id,
-                     sat.subject_token,
-                     sat.latest_message_id,
-                     saml.name"
-
-            query="SELECT
-                     sam.thread_id AS id,
-                     sat.subject_token AS name,
-                     sat.latest_message_id,
-                     saml.name AS mailinglist,
-                     array_to_string(array_agg(sam.body), ' ') AS content
-                   FROM
-                     super_archives_message AS sam
-                     JOIN super_archives_thread AS sat
-                       ON sat.id = sam.thread_id
-                     JOIN super_archives_mailinglist AS saml
-                       ON sat.mailinglist_id = saml.id
-                   GROUP BY
-                     sam.thread_id,
-                     sat.subject_token,
-                     sat.latest_message_id,
-                     saml.name">
-
-      <!--
-        Check about "DISTINCT ON" here: 
-          http://archives.postgresql.org/pgsql-general/2002-06/msg01330.php
-      -->
-      <entity name="first_message"
-              dataSource="colab"
-              transformer="TemplateTransformer"
-              query="SELECT DISTINCT ON (sam.thread_id)
-                       sam.body AS description,
-                       sam.received_time AS created,
-                       sam.subject_clean AS subject,
-                       au.username AS creator
-                     FROM 
-                       super_archives_message AS sam
-                       JOIN super_archives_emailaddress AS saea
-                         ON sam.from_address_id = saea.id
-                       LEFT JOIN auth_user AS au
-                         ON au.id = saea.user_id
-                     WHERE
-                       sam.thread_id = ${thread.id}
-                     ORDER BY 
-                       sam.thread_id,
-                       sam.received_time">
-        <field column="title" template="${first_message.subject}" />
-      </entity>
-
-      <entity name="latest_message"
-              dataSource="colab"
-              query="SELECT 
-                       received_time AS modified
-                     FROM 
-                       super_archives_message
-                     WHERE
-                       id = ${thread.latest_message_id}" />
-
-      <entity name="thread_collaborators"
-              dataSource="colab"
-              query="SELECT DISTINCT 
-                       au.username AS collaborator
-                     FROM 
-                       super_archives_message AS sam
-                       JOIN super_archives_emailaddress AS saea
-                         ON sam.from_address_id = saea.id
-                       JOIN auth_user AS au
-                         ON au.id = saea.user_id
-                     WHERE
-                       thread_id = ${thread.id}" />
-
-      <field column="uid" template="THREAD_${thread.id}" />
-      <field column="type" template="thread" />
-      <field column="created" name="created" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss" />
-      <field column="modified" name="modified" 
-             dateTimeFormat="yyyy-MM-dd hh:mm:ss" />
-    </entity> 
-  </document>
-          
-</dataConfig>
-  
-<!-- 
-vim: ts=2 sw=2 ss=2 expandtab:
--->
@@ -1,499 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" ?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements.  See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License.  You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<!--  
- This is the Solr schema file. This file should be named "schema.xml" and
- should be in the conf directory under the solr home
- (i.e. ./solr/conf/schema.xml by default) 
- or located where the classloader for the Solr webapp can find it.
-
- This example schema is the recommended starting point for users.
- It should be kept correct and concise, usable out-of-the-box.
-
- For more information, on how to customize this file, please see
- http://wiki.apache.org/solr/SchemaXml
-
- PERFORMANCE NOTE: this schema includes many optional features and should not
- be used for benchmarking.  To improve performance one could
-  - set stored="false" for all fields possible (esp large fields) when you
-    only need to search on the field but don't need to return the original
-    value.
-  - set indexed="false" if you don't need to search on the field, but only
-    return the field as a result of searching on other indexed fields.
-  - remove all unneeded copyField statements
-  - for best index size and searching performance, set "index" to false
-    for all general text fields, use copyField to copy them to the
-    catchall "text" field, and use that for searching.
-  - For maximum indexing performance, use the StreamingUpdateSolrServer
-    java client.
-  - Remember to run the JVM in server mode, and use a higher logging level
-    that avoids logging every request
--->
-
-<schema name="example" version="1.2">
-  <!-- attribute "name" is the name of this schema and is only used for display purposes.
-       Applications should change this to reflect the nature of the search collection.
-       version="1.2" is Solr's version number for the schema syntax and semantics.  It should
-       not normally be changed by applications.
-       1.0: multiValued attribute did not exist, all fields are multiValued by nature
-       1.1: multiValued attribute introduced, false by default 
-       1.2: omitTermFreqAndPositions attribute introduced, true by default except for text fields.
-     -->
-
-  <types>
-    <!-- field type definitions. The "name" attribute is
-       just a label to be used by field definitions.  The "class"
-       attribute and any other attributes determine the real
-       behavior of the fieldType.
-         Class names starting with "solr" refer to java classes in the
-       org.apache.solr.analysis package.
-    -->
-
-    <!-- The StrField type is not analyzed, but indexed/stored verbatim.  
-       - StrField and TextField support an optional compressThreshold which
-       limits compression (if enabled in the derived fields) to values which
-       exceed a certain size (in characters).
-    -->
-    <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
-
-    <!-- boolean type: "true" or "false" -->
-    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
-    <!--Binary data type. The data should be sent/retrieved in as Base64 encoded Strings -->
-    <fieldtype name="binary" class="solr.BinaryField"/>
-
-    <!-- The optional sortMissingLast and sortMissingFirst attributes are
-         currently supported on types that are sorted internally as strings.
-	       This includes "string","boolean","sint","slong","sfloat","sdouble","pdate"
-       - If sortMissingLast="true", then a sort on this field will cause documents
-         without the field to come after documents with the field,
-         regardless of the requested sort order (asc or desc).
-       - If sortMissingFirst="true", then a sort on this field will cause documents
-         without the field to come before documents with the field,
-         regardless of the requested sort order.
-       - If sortMissingLast="false" and sortMissingFirst="false" (the default),
-         then default lucene sorting will be used which places docs without the
-         field first in an ascending sort and last in a descending sort.
-    -->    
-
-    <!--
-      Default numeric field types. For faster range queries, consider the tint/tfloat/tlong/tdouble types.
-    -->
-    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
-
-    <!--
-     Numeric field types that index each value at various levels of precision
-     to accelerate range queries when the number of values between the range
-     endpoints is large. See the javadoc for NumericRangeQuery for internal
-     implementation details.
-
-     Smaller precisionStep values (specified in bits) will lead to more tokens
-     indexed per value, slightly larger index size, and faster range queries.
-     A precisionStep of 0 disables indexing at different precision levels.
-    -->
-    <fieldType name="tint" class="solr.TrieIntField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="tlong" class="solr.TrieLongField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
-    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
-
-    <!-- The format for this date field is of the form 1995-12-31T23:59:59Z, and
-         is a more restricted form of the canonical representation of dateTime
-         http://www.w3.org/TR/xmlschema-2/#dateTime    
-         The trailing "Z" designates UTC time and is mandatory.
-         Optional fractional seconds are allowed: 1995-12-31T23:59:59.999Z
-         All other components are mandatory.
-
-         Expressions can also be used to denote calculations that should be
-         performed relative to "NOW" to determine the value, ie...
-
-               NOW/HOUR
-                  ... Round to the start of the current hour
-               NOW-1DAY
-                  ... Exactly 1 day prior to now
-               NOW/DAY+6MONTHS+3DAYS
-                  ... 6 months and 3 days in the future from the start of
-                      the current day
-                      
-         Consult the DateField javadocs for more information.
-
-         Note: For faster range queries, consider the tdate type
-      -->
-    <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
-
-    <!-- A Trie based date field for faster date range queries and date faceting. -->
-    <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
-
-
-    <!--
-      Note:
-      These should only be used for compatibility with existing indexes (created with older Solr versions)
-      or if "sortMissingFirst" or "sortMissingLast" functionality is needed. Use Trie based fields instead.
-
-      Plain numeric field types that store and index the text
-      value verbatim (and hence don't support range queries, since the
-      lexicographic ordering isn't equal to the numeric ordering)
-    -->
-    <fieldType name="pint" class="solr.IntField" omitNorms="true"/>
-    <fieldType name="plong" class="solr.LongField" omitNorms="true"/>
-    <fieldType name="pfloat" class="solr.FloatField" omitNorms="true"/>
-    <fieldType name="pdouble" class="solr.DoubleField" omitNorms="true"/>
-    <fieldType name="pdate" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
-
-
-    <!--
-      Note:
-      These should only be used for compatibility with existing indexes (created with older Solr versions)
-      or if "sortMissingFirst" or "sortMissingLast" functionality is needed. Use Trie based fields instead.
-
-      Numeric field types that manipulate the value into
-      a string value that isn't human-readable in its internal form,
-      but with a lexicographic ordering the same as the numeric ordering,
-      so that range queries work correctly.
-    -->
-    <fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
-    <fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
-    <fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
-    <fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>
-
-
-    <!-- The "RandomSortField" is not used to store or search any
-         data.  You can declare fields of this type it in your schema
-         to generate pseudo-random orderings of your docs for sorting 
-         purposes.  The ordering is generated based on the field name 
-         and the version of the index, As long as the index version
-         remains unchanged, and the same field name is reused,
-         the ordering of the docs will be consistent.  
-         If you want different psuedo-random orderings of documents,
-         for the same version of the index, use a dynamicField and
-         change the name
-     -->
-    <fieldType name="random" class="solr.RandomSortField" indexed="true" />
-
-    <!-- solr.TextField allows the specification of custom text analyzers
-         specified as a tokenizer and a list of token filters. Different
-         analyzers may be specified for indexing and querying.
-
-         The optional positionIncrementGap puts space between multiple fields of
-         this type on the same document, with the purpose of preventing false phrase
-         matching across fields.
-
-         For more info on customizing your analyzer chain, please see
-         http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
-     -->
-
-    <!-- One can also specify an existing Analyzer class that has a
-         default constructor via the class attribute on the analyzer element
-    <fieldType name="text_greek" class="solr.TextField">
-      <analyzer class="org.apache.lucene.analysis.el.GreekAnalyzer"/>
-    </fieldType>
-    -->
-
-    <!-- A text field that only splits on whitespace for exact matching of words -->
-    <fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
-      <analyzer>
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-      </analyzer>
-    </fieldType>
-
-    <!-- A text field that uses WordDelimiterFilter to enable splitting and matching of
-        words on case-change, alpha numeric boundaries, and non-alphanumeric chars,
-        so that a query of "wifi" or "wi fi" could match a document containing "Wi-Fi".
-        Synonyms and stopwords are customized by external files, and stemming is enabled.
-        -->
-    <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
-      <analyzer type="index">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <!-- in this example, we will only use synonyms at query time
-        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
-        -->
-        <!-- Case insensitive stop word removal.
-          add enablePositionIncrements=true in both the index and query
-          analyzers to leave a 'gap' for more accurate phrase queries.
-        -->
-        <filter class="solr.StopFilterFactory"
-                ignoreCase="true"
-                words="stopwords.txt"
-                enablePositionIncrements="true"
-                />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-        <filter class="solr.SnowballPorterFilterFactory" language="Portuguese" protected="protwords.txt"/>
-      </analyzer>
-      <analyzer type="query">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
-        <filter class="solr.StopFilterFactory"
-                ignoreCase="true"
-                words="stopwords.txt"
-                enablePositionIncrements="true"
-                />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-        <filter class="solr.SnowballPorterFilterFactory" language="Portuguese" protected="protwords.txt"/>
-      </analyzer>
-    </fieldType>
-
-
-    <!-- Less flexible matching, but less false matches.  Probably not ideal for product names,
-         but may be good for SKUs.  Can insert dashes in the wrong place and still match. -->
-    <fieldType name="textTight" class="solr.TextField" positionIncrementGap="100" >
-      <analyzer>
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
-        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="0" generateNumberParts="0" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-        <filter class="solr.SnowballPorterFilterFactory" language="Portuguese" protected="protwords.txt"/>
-        <!-- this filter can remove any duplicate tokens that appear at the same position - sometimes
-             possible with WordDelimiterFilter in conjuncton with stemming. -->
-        <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
-      </analyzer>
-    </fieldType>
-
-
-    <!-- A general unstemmed text field - good if one does not know the language of the field -->
-    <fieldType name="textgen" class="solr.TextField" positionIncrementGap="100">
-      <analyzer type="index">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-      </analyzer>
-      <analyzer type="query">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
-        <filter class="solr.StopFilterFactory"
-                ignoreCase="true"
-                words="stopwords.txt"
-                enablePositionIncrements="true"
-                />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-      </analyzer>
-    </fieldType>
-
-
-    <!-- A general unstemmed text field that indexes tokens normally and also
-         reversed (via ReversedWildcardFilterFactory), to enable more efficient 
-	 leading wildcard queries. -->
-    <fieldType name="text_rev" class="solr.TextField" positionIncrementGap="100">
-      <analyzer type="index">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-        <filter class="solr.ReversedWildcardFilterFactory" withOriginal="true"
-           maxPosAsterisk="3" maxPosQuestion="2" maxFractionAsterisk="0.33"/>
-      </analyzer>
-      <analyzer type="query">
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
-        <filter class="solr.StopFilterFactory"
-                ignoreCase="true"
-                words="stopwords.txt"
-                enablePositionIncrements="true"
-                />
-        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>
-        <filter class="solr.LowerCaseFilterFactory"/>
-      </analyzer>
-    </fieldType>
-
-    <!-- charFilter + WhitespaceTokenizer  -->
-    <!--
-    <fieldType name="textCharNorm" class="solr.TextField" positionIncrementGap="100" >
-      <analyzer>
-        <charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-      </analyzer>
-    </fieldType>
-    -->
-
-    <!-- This is an example of using the KeywordTokenizer along
-         With various TokenFilterFactories to produce a sortable field
-         that does not include some properties of the source text
-      -->
-    <fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
-      <analyzer>
-        <!-- KeywordTokenizer does no actual tokenizing, so the entire
-             input string is preserved as a single token
-          -->
-        <tokenizer class="solr.KeywordTokenizerFactory"/>
-        <!-- The LowerCase TokenFilter does what you expect, which can be
-             when you want your sorting to be case insensitive
-          -->
-        <filter class="solr.LowerCaseFilterFactory" />
-        <!-- The TrimFilter removes any leading or trailing whitespace -->
-        <filter class="solr.TrimFilterFactory" />
-        <!-- The PatternReplaceFilter gives you the flexibility to use
-             Java Regular expression to replace any sequence of characters
-             matching a pattern with an arbitrary replacement string, 
-             which may include back references to portions of the original
-             string matched by the pattern.
-             
-             See the Java Regular Expression documentation for more
-             information on pattern and replacement string syntax.
-             
-             http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.html
-          -->
-        <filter class="solr.PatternReplaceFilterFactory"
-                pattern="([^a-z])" replacement="" replace="all"
-        />
-      </analyzer>
-    </fieldType>
-    
-    <fieldtype name="phonetic" stored="false" indexed="true" class="solr.TextField" >
-      <analyzer>
-        <tokenizer class="solr.StandardTokenizerFactory"/>
-        <filter class="solr.DoubleMetaphoneFilterFactory" inject="false"/>
-      </analyzer>
-    </fieldtype>
-
-    <fieldtype name="payloads" stored="false" indexed="true" class="solr.TextField" >
-      <analyzer>
-        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
-        <!--
-        The DelimitedPayloadTokenFilter can put payloads on tokens... for example,
-        a token of "foo|1.4"  would be indexed as "foo" with a payload of 1.4f
-        Attributes of the DelimitedPayloadTokenFilterFactory : 
-         "delimiter" - a one character delimiter. Default is | (pipe)
-	 "encoder" - how to encode the following value into a playload
-	    float -> org.apache.lucene.analysis.payloads.FloatEncoder,
-	    integer -> o.a.l.a.p.IntegerEncoder
-	    identity -> o.a.l.a.p.IdentityEncoder
-            Fully Qualified class name implementing PayloadEncoder, Encoder must have a no arg constructor.
-         -->
-        <filter class="solr.DelimitedPayloadTokenFilterFactory" encoder="float"/>
-      </analyzer>
-    </fieldtype>
-
-    <!-- lowercases the entire field value, keeping it as a single token.  -->
-    <fieldType name="lowercase" class="solr.TextField" positionIncrementGap="100">
-      <analyzer>
-        <tokenizer class="solr.KeywordTokenizerFactory"/>
-        <filter class="solr.LowerCaseFilterFactory" />
-      </analyzer>
-    </fieldType>
-
-
-    <!-- since fields of this type are by default not stored or indexed,
-         any data added to them will be ignored outright.  --> 
-    <fieldtype name="ignored" stored="false" indexed="false" multiValued="true" class="solr.StrField" /> 
-
- </types>
-
-
- <fields>
-   <!-- Valid attributes for fields:
-     name: mandatory - the name for the field
-     type: mandatory - the name of a previously defined type from the 
-       <types> section
-     indexed: true if this field should be indexed (searchable or sortable)
-     stored: true if this field should be retrievable
-     compressed: [false] if this field should be stored using gzip compression
-       (this will only apply if the field type is compressable; among
-       the standard field types, only TextField and StrField are)
-     multiValued: true if this field may contain multiple values per document
-     omitNorms: (expert) set to true to omit the norms associated with
-       this field (this disables length normalization and index-time
-       boosting for the field, and saves some memory).  Only full-text
-       fields or fields that need an index-time boost need norms.
-     termVectors: [false] set to true to store the term vector for a
-       given field.
-       When using MoreLikeThis, fields used for similarity should be
-       stored for best performance.
-     termPositions: Store position information with the term vector.  
-       This will increase storage costs.
-     termOffsets: Store offset information with the term vector. This 
-       will increase storage costs.
-     default: a value that should be used if no value is specified
-       when adding a document.
-   -->
-   
-   <!--  Base fields (all should be indexed and stored)-->
-   <field name="uid" type="string" indexed="true" stored="true" required="true" /> 
-   <field name="id" type="string" indexed="true" stored="true" required="true" />
-   <field name="type" type="string" indexed="true" stored="true" required="true" />
-   <field name="title" type="text" indexed="true" stored="true" required="true" />
-   <field name="description" type="text" indexed="true" stored="true" />
-   <field name="creator" type="string" indexed="true" stored="true" />
-   <field name="created" type="date" indexed="true" stored="true" />
-   <field name="modified" type="date" indexed="true" stored="true" />
-  
-   <!-- All next fields shoult NOT be stored --> 
-   <field name="collaborator" type="textgen" indexed="true" stored="false" multiValued="true" />
-   <field name="name" type="string" indexed="true" stored="false" />
-   <field name="content" type="text" indexed="true" stored="false" />
-   <field name="comment" type="text" indexed="true" stored="false" multiValued="true" />
-   <field name="keyword" type="text" indexed="true" stored="false" multiValued="true" />
-   <field name="milestone" type="string" indexed="true" stored="false" />
-   <field name="priority" type="string" indexed="true" stored="false" />
-   <field name="component" type="string" indexed="true" stored="false" />
-   <field name="version" type="string" indexed="true" stored="false" />
-   <field name="severity" type="string" indexed="true" stored="false" />
-   <field name="reporter" type="textgen" indexed="true" stored="false" />
-   <field name="owner" type="textgen" indexed="true" stored="false" /> 
-   <field name="status" type="string" indexed="true" stored="false" /> 
-   <field name="subject" type="text" indexed="true" stored="false" />
-   <field name="revision" type="int" indexed="true" stored="false" />
-   <field name="mailinglist" type="textgen" indexed="true" stored="false" />
-
-   <!-- catchall field, containing all other searchable text fields (implemented
-        via copyField further on in this schema  -->
-   <field name="default" type="textgen" indexed="true" stored="false" multiValued="true"/>
- </fields>
-
- <!-- Field to use to determine and enforce document uniqueness. 
-      Unless this field is marked with required="false", it will be a required field
-   -->
- <uniqueKey>uid</uniqueKey>
-
- <!-- field for the QueryParser to use when an explicit fieldname is absent -->
- <defaultSearchField>default</defaultSearchField>
-
- <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
- <solrQueryParser defaultOperator="OR"/>
-
-  <!-- copyField commands copy one field to another at the time a document
-        is added to the index.  It's used either to index the same field differently,
-        or to add multiple fields to the same field for easier/faster searching.  -->
-
-   <copyField source="content" dest="default" />
-   <copyField source="title" dest="default" />
-   <copyField source="description" dest="default" />
-   <copyField source="creator" dest="default" />
-   <copyField source="collaborator" dest="default" />
-   <copyField source="comment" dest="default" />
-   <copyField source="keyword" dest="default" />
- 
-<!-- Similarity is the scoring routine for each document vs. a query.
-      A custom similarity may be specified here, but the default is fine
-      for most applications.  -->
- <!-- <similarity class="org.apache.lucene.search.DefaultSimilarity"/> -->
- <!-- ... OR ...
-      Specify a SimilarityFactory class name implementation
-      allowing parameters to be used.
- -->
- <!--
- <similarity class="com.example.solr.CustomSimilarityFactory">
-   <str name="paramkey">param value</str>
- </similarity>
- -->
-
-</schema>
@@ -1,1040 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" ?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements.  See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License.  You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<!-- 
-     For more details about configurations options that may appear in this 
-     file, see http://wiki.apache.org/solr/SolrConfigXml.
-
-     Specifically, the Solr Config can support XInclude, which may make it easier to manage
-     the configuration.  See https://issues.apache.org/jira/browse/SOLR-1167
--->
-<config>
-  <!-- Set this to 'false' if you want solr to continue working after it has 
-       encountered an severe configuration error.  In a production environment, 
-       you may want solr to keep working even if one handler is mis-configured.
-
-       You may also set this to false using by setting the system property:
-         -Dsolr.abortOnConfigurationError=false
-     -->
-  <abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
-
-  <!-- lib directives can be used to instruct Solr to load an Jars identified
-       and use them to resolve any "plugins" specified in your solrconfig.xml or
-       schema.xml (ie: Analyzers, Request Handlers, etc...).
-
-       All directories and paths are resolved relative the instanceDir.
-
-       If a "./lib" directory exists in your instanceDir, all files found in it
-       are included as if you had used the following syntax...
-       
-              <lib dir="./lib" />
-    -->
-  <!-- A dir option by itself adds any files found in the directory to the
-       classpath, this is useful for including all jars in a directory.
-    -->
-  <!--lib dir="../../contrib/extraction/lib" /-->
-  <!-- When a regex is specified in addition to a directory, only the files in that
-       directory which completely match the regex (anchored on both ends)
-       will be included.
-    -->
-  <!--lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
-  <lib dir="../../dist/" regex="apache-solr-clustering-\d.*\.jar" /-->
-  <!-- If a dir option (with or without a regex) is used and nothing is found
-       that matches, it will be ignored
-    -->
-  <!--lib dir="../../contrib/clustering/lib/downloads/" />
-  <lib dir="../../contrib/clustering/lib/" />
-  <lib dir="/total/crap/dir/ignored" /--> 
-  <!-- an exact path can be used to specify a specific file.  This will cause
-       a serious error to be logged if it can't be loaded.
-  <lib path="../a-jar-that-does-not-exist.jar" /> 
-  -->
-
-  
-  <!-- Used to specify an alternate directory to hold all index data
-       other than the default ./data under the Solr home.
-       If replication is in use, this should match the replication configuration. -->
-  <dataDir>/var/lib/solr/data</dataDir>
-
-
-  <!-- WARNING: this <indexDefaults> section only provides defaults for index writers
-       in general. See also the <mainIndex> section after that when changing parameters
-       for Solr's main Lucene index. -->
-  <indexDefaults>
-   <!-- Values here affect all index writers and act as a default unless overridden. -->
-    <useCompoundFile>false</useCompoundFile>
-
-    <mergeFactor>10</mergeFactor>
-    <!-- If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush
-     based on whichever limit is hit first.  -->
-    <!--<maxBufferedDocs>1000</maxBufferedDocs>-->
-
-    <!-- Sets the amount of RAM that may be used by Lucene indexing
-      for buffering added documents and deletions before they are
-      flushed to the Directory.  -->
-    <ramBufferSizeMB>32</ramBufferSizeMB>
-    <!-- <maxMergeDocs>2147483647</maxMergeDocs> -->
-    <maxFieldLength>10000</maxFieldLength>
-    <writeLockTimeout>1000</writeLockTimeout>
-    <commitLockTimeout>10000</commitLockTimeout>
-
-    <!--
-     Expert: Turn on Lucene's auto commit capability.  This causes intermediate
-     segment flushes to write a new lucene index descriptor, enabling it to be
-     opened by an external IndexReader.  This can greatly slow down indexing
-     speed.  NOTE: Despite the name, this value does not have any relation to
-     Solr's autoCommit functionality
-     -->
-    <!--<luceneAutoCommit>false</luceneAutoCommit>-->
-
-    <!--
-     Expert: The Merge Policy in Lucene controls how merging is handled by
-     Lucene.  The default in 2.3 is the LogByteSizeMergePolicy, previous
-     versions used LogDocMergePolicy.
-
-     LogByteSizeMergePolicy chooses segments to merge based on their size.  The
-     Lucene 2.2 default, LogDocMergePolicy chose when to merge based on number
-     of documents
-
-     Other implementations of MergePolicy must have a no-argument constructor
-     -->
-    <!--<mergePolicy class="org.apache.lucene.index.LogByteSizeMergePolicy"/>-->
-
-    <!--
-     Expert:
-     The Merge Scheduler in Lucene controls how merges are performed.  The
-     ConcurrentMergeScheduler (Lucene 2.3 default) can perform merges in the
-     background using separate threads.  The SerialMergeScheduler (Lucene 2.2
-     default) does not.
-     -->
-    <!--<mergeScheduler class="org.apache.lucene.index.ConcurrentMergeScheduler"/>-->
-
-	  
-    <!--
-      This option specifies which Lucene LockFactory implementation to use.
-      
-      single = SingleInstanceLockFactory - suggested for a read-only index
-               or when there is no possibility of another process trying
-               to modify the index.
-      native = NativeFSLockFactory  - uses OS native file locking
-      simple = SimpleFSLockFactory  - uses a plain file for locking
-
-      (For backwards compatibility with Solr 1.2, 'simple' is the default
-       if not specified.)
-    -->
-    <lockType>native</lockType>
-    <!--
-     Expert:
-    Controls how often Lucene loads terms into memory -->
-    <!--<termIndexInterval>256</termIndexInterval>-->
-  </indexDefaults>
-
-  <mainIndex>
-    <!-- options specific to the main on-disk lucene index -->
-    <useCompoundFile>false</useCompoundFile>
-    <ramBufferSizeMB>32</ramBufferSizeMB>
-    <mergeFactor>10</mergeFactor>
-    <!-- Deprecated -->
-    <!--<maxBufferedDocs>1000</maxBufferedDocs>-->
-    <!--<maxMergeDocs>2147483647</maxMergeDocs>-->
-
-    <!-- inherit from indexDefaults <maxFieldLength>10000</maxFieldLength> -->
-
-    <!-- If true, unlock any held write or commit locks on startup. 
-         This defeats the locking mechanism that allows multiple
-         processes to safely access a lucene index, and should be
-         used with care.
-         This is not needed if lock type is 'none' or 'single'
-     -->
-    <unlockOnStartup>false</unlockOnStartup>
-    
-    <!-- If true, IndexReaders will be reopened (often more efficient) instead
-         of closed and then opened.  -->
-    <reopenReaders>true</reopenReaders>
-
-    <!--
-     Expert:
-    Controls how often Lucene loads terms into memory.  Default is 128 and is likely good for most everyone. -->
-    <!--<termIndexInterval>256</termIndexInterval>-->
-
-    <!--
-        Custom deletion policies can specified here. The class must
-        implement org.apache.lucene.index.IndexDeletionPolicy.
-
-        http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/index/IndexDeletionPolicy.html
-
-        The standard Solr IndexDeletionPolicy implementation supports deleting
-        index commit points on number of commits, age of commit point and
-        optimized status.
-
-        The latest commit point should always be preserved regardless
-        of the criteria.
-    -->
-    <deletionPolicy class="solr.SolrDeletionPolicy">
-      <!-- The number of commit points to be kept -->
-      <str name="maxCommitsToKeep">1</str>
-      <!-- The number of optimized commit points to be kept -->
-      <str name="maxOptimizedCommitsToKeep">0</str>
-      <!--
-          Delete all commit points once they have reached the given age.
-          Supports DateMathParser syntax e.g.
-          
-          <str name="maxCommitAge">30MINUTES</str>
-          <str name="maxCommitAge">1DAY</str>
-      -->
-    </deletionPolicy>
-
-    <!--  To aid in advanced debugging, you may turn on IndexWriter debug logging.
-      Setting to true will set the file that the underlying Lucene IndexWriter
-      will write its debug infostream to.  -->
-     <infoStream file="INFOSTREAM.txt">false</infoStream> 
-
-  </mainIndex>
-
-  <!--	Enables JMX if and only if an existing MBeanServer is found, use this
-    if you want to configure JMX through JVM parameters. Remove this to disable
-    exposing Solr configuration and statistics to JMX.
-
-		If you want to connect to a particular server, specify the agentId
-		e.g. <jmx agentId="myAgent" />
-
-		If you want to start a new MBeanServer, specify the serviceUrl
-		e.g <jmx serviceUrl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr"/>
-
-		For more details see http://wiki.apache.org/solr/SolrJmx
-  -->
-  <jmx />
-
-  <!-- the default high-performance update handler -->
-  <updateHandler class="solr.DirectUpdateHandler2">
-    <!-- A prefix of "solr." for class names is an alias that
-         causes solr to search appropriate packages, including
-         org.apache.solr.(search|update|request|core|analysis)
-     -->
-
-    <!-- Perform a <commit/> automatically under certain conditions:
-         maxDocs - number of updates since last commit is greater than this
-         maxTime - oldest uncommited update (in ms) is this long ago
-         Instead of enabling autoCommit, consider using "commitWithin"
-         when adding documents. http://wiki.apache.org/solr/UpdateXmlMessages
-    <autoCommit> 
-      <maxDocs>10000</maxDocs>
-      <maxTime>1000</maxTime> 
-    </autoCommit>
-    -->
-
-
-    <!-- The RunExecutableListener executes an external command from a
-      hook such as postCommit or postOptimize.
-         exe - the name of the executable to run
-         dir - dir to use as the current working directory. default="."
-         wait - the calling thread waits until the executable returns. default="true"
-         args - the arguments to pass to the program.  default=nothing
-         env - environment variables to set.  default=nothing
-      -->
-    <!-- A postCommit event is fired after every commit or optimize command
-    <listener event="postCommit" class="solr.RunExecutableListener">
-      <str name="exe">solr/bin/snapshooter</str>
-      <str name="dir">.</str>
-      <bool name="wait">true</bool>
-      <arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
-      <arr name="env"> <str>MYVAR=val1</str> </arr>
-    </listener>
-    -->
-    <!-- A postOptimize event is fired only after every optimize command
-    <listener event="postOptimize" class="solr.RunExecutableListener">
-      <str name="exe">snapshooter</str>
-      <str name="dir">solr/bin</str>
-      <bool name="wait">true</bool>
-    </listener>
-    -->
-
-  </updateHandler>
-  
-  <!-- Use the following format to specify a custom IndexReaderFactory - allows for alternate
-       IndexReader implementations.
-
-       ** Experimental Feature **
-       Please note - Using a custom IndexReaderFactory may prevent certain other features
-       from working. The API to IndexReaderFactory may change without warning or may even
-       be removed from future releases if the problems cannot be resolved.
-
-       ** Features that may not work with custom IndexReaderFactory **
-       The ReplicationHandler assumes a disk-resident index. Using a custom
-       IndexReader implementation may cause incompatibility with ReplicationHandler and
-       may cause replication to not work correctly. See SOLR-1366 for details.
-
-  <indexReaderFactory name="IndexReaderFactory" class="package.class">
-    Parameters as required by the implementation
-  </indexReaderFactory >
-  -->
-  <!-- To set the termInfosIndexDivisor, do this: -->
-  <!--<indexReaderFactory name="IndexReaderFactory" class="org.apache.solr.core.StandardIndexReaderFactory">
-    <int name="termInfosIndexDivisor">12</int>
-  </indexReaderFactory >-->
-
-
-  <query>
-    <!-- Maximum number of clauses in a boolean query... in the past, this affected
-        range or prefix queries that expanded to big boolean queries - built in Solr 
-        query parsers no longer create queries with this limitation. 
-        An exception is thrown if exceeded.  -->
-    <maxBooleanClauses>1024</maxBooleanClauses>
-
-
-    <!-- There are two implementations of cache available for Solr,
-         LRUCache, based on a synchronized LinkedHashMap, and
-         FastLRUCache, based on a ConcurrentHashMap.  FastLRUCache has faster gets
-         and slower puts in single threaded operation and thus is generally faster
-         than LRUCache when the hit ratio of the cache is high (> 75%), and may be
-         faster under other scenarios on multi-cpu systems. -->
-    <!-- Cache used by SolrIndexSearcher for filters (DocSets),
-         unordered sets of *all* documents that match a query.
-         When a new searcher is opened, its caches may be prepopulated
-         or "autowarmed" using data from caches in the old searcher.
-         autowarmCount is the number of items to prepopulate.  For LRUCache,
-         the autowarmed items will be the most recently accessed items.
-       Parameters:
-         class - the SolrCache implementation LRUCache or FastLRUCache
-         size - the maximum number of entries in the cache
-         initialSize - the initial capacity (number of entries) of
-           the cache.  (seel java.util.HashMap)
-         autowarmCount - the number of entries to prepopulate from
-           and old cache.
-         -->
-    <filterCache
-      class="solr.FastLRUCache"
-      size="512"
-      initialSize="512"
-      autowarmCount="0"/>
-
-    <!-- Cache used to hold field values that are quickly accessible
-         by document id.  The fieldValueCache is created by default
-         even if not configured here.
-      <fieldValueCache
-        class="solr.FastLRUCache"
-        size="512"
-        autowarmCount="128"
-        showItems="32"
-      />
-    -->
-
-   <!-- queryResultCache caches results of searches - ordered lists of
-         document ids (DocList) based on a query, a sort, and the range
-         of documents requested.  -->
-    <queryResultCache
-      class="solr.LRUCache"
-      size="512"
-      initialSize="512"
-      autowarmCount="0"/>
-
-  <!-- documentCache caches Lucene Document objects (the stored fields for each document).
-       Since Lucene internal document ids are transient, this cache will not be autowarmed.  -->
-    <documentCache
-      class="solr.LRUCache"
-      size="512"
-      initialSize="512"
-      autowarmCount="0"/>
-
-    <!-- If true, stored fields that are not requested will be loaded lazily.
-      This can result in a significant speed improvement if the usual case is to
-      not load all stored fields, especially if the skipped fields are large
-      compressed text fields.
-    -->
-    <enableLazyFieldLoading>true</enableLazyFieldLoading>
-
-    <!-- Example of a generic cache.  These caches may be accessed by name
-         through SolrIndexSearcher.getCache(),cacheLookup(), and cacheInsert().
-         The purpose is to enable easy caching of user/application level data.
-         The regenerator argument should be specified as an implementation
-         of solr.search.CacheRegenerator if autowarming is desired.  -->
-    <!--
-    <cache name="myUserCache"
-      class="solr.LRUCache"
-      size="4096"
-      initialSize="1024"
-      autowarmCount="1024"
-      regenerator="org.mycompany.mypackage.MyRegenerator"
-      />
-    -->
-
-   <!-- An optimization that attempts to use a filter to satisfy a search.
-         If the requested sort does not include score, then the filterCache
-         will be checked for a filter matching the query. If found, the filter
-         will be used as the source of document ids, and then the sort will be
-         applied to that.
-    <useFilterForSortedQuery>true</useFilterForSortedQuery>
-   -->
-
-   <!-- An optimization for use with the queryResultCache.  When a search
-         is requested, a superset of the requested number of document ids
-         are collected.  For example, if a search for a particular query
-         requests matching documents 10 through 19, and queryWindowSize is 50,
-         then documents 0 through 49 will be collected and cached.  Any further
-         requests in that range can be satisfied via the cache.  -->
-    <queryResultWindowSize>20</queryResultWindowSize>
-
-    <!-- Maximum number of documents to cache for any entry in the
-         queryResultCache. -->
-    <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
-
-    <!-- a newSearcher event is fired whenever a new searcher is being prepared
-      and there is a current searcher handling requests (aka registered).
-      It can be used to prime certain caches to prevent long request times for
-      certain requests.
-    -->
-    <!-- QuerySenderListener takes an array of NamedList and executes a
-         local query request for each NamedList in sequence. -->
-    <listener event="newSearcher" class="solr.QuerySenderListener">
-      <arr name="queries">
-        <!--
-        <lst> <str name="q">solr</str> <str name="start">0</str> <str name="rows">10</str> </lst>
-        <lst> <str name="q">rocks</str> <str name="start">0</str> <str name="rows">10</str> </lst>
-        <lst><str name="q">static newSearcher warming query from solrconfig.xml</str></lst>
-        -->
-      </arr>
-    </listener>
-
-    <!-- a firstSearcher event is fired whenever a new searcher is being
-         prepared but there is no current registered searcher to handle
-         requests or to gain autowarming data from. -->
-    <listener event="firstSearcher" class="solr.QuerySenderListener">
-      <arr name="queries">
-        <lst> <str name="q">solr rocks</str><str name="start">0</str><str name="rows">10</str></lst>
-        <lst><str name="q">static firstSearcher warming query from solrconfig.xml</str></lst>
-      </arr>
-    </listener>
-
-    <!-- If a search request comes in and there is no current registered searcher,
-         then immediately register the still warming searcher and use it.  If
-         "false" then all requests will block until the first searcher is done
-         warming. -->
-    <useColdSearcher>false</useColdSearcher>
-
-    <!-- Maximum number of searchers that may be warming in the background
-      concurrently.  An error is returned if this limit is exceeded. Recommend
-      1-2 for read-only slaves, higher for masters w/o cache warming. -->
-    <maxWarmingSearchers>2</maxWarmingSearchers>
-
-  </query>
-
-  <!-- 
-    Let the dispatch filter handler /select?qt=XXX
-    handleSelect=true will use consistent error handling for /select and /update
-    handleSelect=false will use solr1.1 style error formatting
-    -->
-  <requestDispatcher handleSelect="true" >
-    <!--Make sure your system has some authentication before enabling remote streaming!  -->
-    <requestParsers enableRemoteStreaming="true" multipartUploadLimitInKB="2048000" />
-
-    <!-- Set HTTP caching related parameters (for proxy caches and clients).
-          
-         To get the behaviour of Solr 1.2 (ie: no caching related headers)
-         use the never304="true" option and do not specify a value for
-         <cacheControl>
-    -->
-    <!-- <httpCaching never304="true"> -->
-    <httpCaching lastModifiedFrom="openTime"
-                 etagSeed="Solr">
-       <!-- lastModFrom="openTime" is the default, the Last-Modified value
-            (and validation against If-Modified-Since requests) will all be
-            relative to when the current Searcher was opened.
-            You can change it to lastModFrom="dirLastMod" if you want the
-            value to exactly corrispond to when the physical index was last
-            modified.
-
-            etagSeed="..." is an option you can change to force the ETag
-            header (and validation against If-None-Match requests) to be
-            differnet even if the index has not changed (ie: when making
-            significant changes to your config file)
-
-            lastModifiedFrom and etagSeed are both ignored if you use the
-            never304="true" option.
-       -->
-       <!-- If you include a <cacheControl> directive, it will be used to
-            generate a Cache-Control header, as well as an Expires header
-            if the value contains "max-age="
-
-            By default, no Cache-Control header is generated.
-
-            You can use the <cacheControl> option even if you have set
-            never304="true"
-       -->
-       <!-- <cacheControl>max-age=30, public</cacheControl> -->
-    </httpCaching>
-  </requestDispatcher>
-
-
-  <!-- requestHandler plugins... incoming queries will be dispatched to the
-     correct handler based on the path or the qt (query type) param.
-     Names starting with a '/' are accessed with the a path equal to the 
-     registered name.  Names without a leading '/' are accessed with:
-      http://host/app/select?qt=name
-     If no qt is defined, the requestHandler that declares default="true"
-     will be used.
-  -->
-  <requestHandler name="standard" class="solr.SearchHandler" default="true">
-    <!-- default values for query parameters -->
-     <lst name="defaults">
-       <str name="echoParams">explicit</str>
-       <!--
-       <int name="rows">10</int>
-       <str name="fl">*</str>
-       <str name="version">2.1</str>
-        -->
-     </lst>
-  </requestHandler>
-
-<!-- Please refer to http://wiki.apache.org/solr/SolrReplication for details on configuring replication -->
-<!-- remove the <lst name="master"> section if this is just a slave -->
-<!-- remove  the <lst name="slave"> section if this is just a master -->
-<!--
-<requestHandler name="/replication" class="solr.ReplicationHandler" >
-    <lst name="master">
-      <str name="replicateAfter">commit</str>
-      <str name="replicateAfter">startup</str>
-      <str name="confFiles">schema.xml,stopwords.txt</str>
-    </lst>
-    <lst name="slave">
-      <str name="masterUrl">http://localhost:8983/solr/replication</str>
-      <str name="pollInterval">00:00:60</str>
-    </lst>
-</requestHandler>-->
-
-  <!-- DisMaxRequestHandler allows easy searching across multiple fields
-       for simple user-entered phrases.  It's implementation is now
-       just the standard SearchHandler with a default query type
-       of "dismax". 
-       see http://wiki.apache.org/solr/DisMaxRequestHandler
-   -->
-  <requestHandler name="dismax" class="solr.SearchHandler" >
-    <lst name="defaults">
-     <str name="defType">dismax</str>
-     <str name="echoParams">explicit</str>
-     <float name="tie">0.01</float>
-     <str name="qf">
-        text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
-     </str>
-     <str name="pf">
-        text^0.2 features^1.1 name^1.5 manu^1.4 manu_exact^1.9
-     </str>
-     <str name="bf">
-        popularity^0.5 recip(price,1,1000,1000)^0.3
-     </str>
-     <str name="fl">
-        id,name,price,score
-     </str>
-     <str name="mm">
-        2&lt;-1 5&lt;-2 6&lt;90%
-     </str>
-     <int name="ps">100</int>
-     <str name="q.alt">*:*</str>
-     <!-- example highlighter config, enable per-query with hl=true -->
-     <str name="hl.fl">text features name</str>
-     <!-- for this field, we want no fragmenting, just highlighting -->
-     <str name="f.name.hl.fragsize">0</str>
-     <!-- instructs Solr to return the field itself if no query terms are
-          found -->
-     <str name="f.name.hl.alternateField">name</str>
-     <str name="f.text.hl.fragmenter">regex</str> <!-- defined below -->
-    </lst>
-  </requestHandler>
-
-  <!-- Note how you can register the same handler multiple times with
-       different names (and different init parameters)
-    -->
-  <requestHandler name="partitioned" class="solr.SearchHandler" >
-    <lst name="defaults">
-     <str name="defType">dismax</str>
-     <str name="echoParams">explicit</str>
-     <str name="qf">text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0</str>
-     <str name="mm">2&lt;-1 5&lt;-2 6&lt;90%</str>
-     <!-- This is an example of using Date Math to specify a constantly
-          moving date range in a config...
-       -->
-     <str name="bq">incubationdate_dt:[* TO NOW/DAY-1MONTH]^2.2</str>
-    </lst>
-    <!-- In addition to defaults, "appends" params can be specified
-         to identify values which should be appended to the list of
-         multi-val params from the query (or the existing "defaults").
-
-         In this example, the param "fq=instock:true" will be appended to
-         any query time fq params the user may specify, as a mechanism for
-         partitioning the index, independent of any user selected filtering
-         that may also be desired (perhaps as a result of faceted searching).
-
-         NOTE: there is *absolutely* nothing a client can do to prevent these
-         "appends" values from being used, so don't use this mechanism
-         unless you are sure you always want it.
-      -->
-    <lst name="appends">
-      <str name="fq">inStock:true</str>
-    </lst>
-    <!-- "invariants" are a way of letting the Solr maintainer lock down
-         the options available to Solr clients.  Any params values
-         specified here are used regardless of what values may be specified
-         in either the query, the "defaults", or the "appends" params.
-
-         In this example, the facet.field and facet.query params are fixed,
-         limiting the facets clients can use.  Faceting is not turned on by
-         default - but if the client does specify facet=true in the request,
-         these are the only facets they will be able to see counts for;
-         regardless of what other facet.field or facet.query params they
-         may specify.
-
-         NOTE: there is *absolutely* nothing a client can do to prevent these
-         "invariants" values from being used, so don't use this mechanism
-         unless you are sure you always want it.
-      -->
-    <lst name="invariants">
-      <str name="facet.field">cat</str>
-      <str name="facet.field">manu_exact</str>
-      <str name="facet.query">price:[* TO 500]</str>
-      <str name="facet.query">price:[500 TO *]</str>
-    </lst>
-  </requestHandler>
-
-
-  <!--
-   Search components are registered to SolrCore and used by Search Handlers
-   
-   By default, the following components are avaliable:
-    
-   <searchComponent name="query"     class="org.apache.solr.handler.component.QueryComponent" />
-   <searchComponent name="facet"     class="org.apache.solr.handler.component.FacetComponent" />
-   <searchComponent name="mlt"       class="org.apache.solr.handler.component.MoreLikeThisComponent" />
-   <searchComponent name="highlight" class="org.apache.solr.handler.component.HighlightComponent" />
-   <searchComponent name="stats"     class="org.apache.solr.handler.component.StatsComponent" />
-   <searchComponent name="debug"     class="org.apache.solr.handler.component.DebugComponent" />
-   
-   Default configuration in a requestHandler would look like:
-    <arr name="components">
-      <str>query</str>
-      <str>facet</str>
-      <str>mlt</str>
-      <str>highlight</str>
-      <str>stats</str>
-      <str>debug</str>
-    </arr>
-
-    If you register a searchComponent to one of the standard names, that will be used instead.
-    To insert components before or after the 'standard' components, use:
-    
-    <arr name="first-components">
-      <str>myFirstComponentName</str>
-    </arr>
-    
-    <arr name="last-components">
-      <str>myLastComponentName</str>
-    </arr>
-  -->
-
-   <!-- The spell check component can return a list of alternative spelling
-  suggestions.  -->
-  <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
-
-    <str name="queryAnalyzerFieldType">textSpell</str>
-
-    <lst name="spellchecker">
-      <str name="name">default</str>
-      <str name="field">name</str>
-      <str name="spellcheckIndexDir">./spellchecker</str>
-    </lst>
-
-    <!-- a spellchecker that uses a different distance measure
-    <lst name="spellchecker">
-      <str name="name">jarowinkler</str>
-      <str name="field">spell</str>
-      <str name="distanceMeasure">org.apache.lucene.search.spell.JaroWinklerDistance</str>
-      <str name="spellcheckIndexDir">./spellchecker2</str>
-    </lst>
-     -->
-
-    <!-- a file based spell checker
-    <lst name="spellchecker">
-      <str name="classname">solr.FileBasedSpellChecker</str>
-      <str name="name">file</str>
-      <str name="sourceLocation">spellings.txt</str>
-      <str name="characterEncoding">UTF-8</str>
-      <str name="spellcheckIndexDir">./spellcheckerFile</str>
-    </lst>
-    -->
-  </searchComponent>
-
-  <!-- A request handler utilizing the spellcheck component.  
-  #############################################################################
-  NOTE: This is purely as an example.  The whole purpose of the
-  SpellCheckComponent is to hook it into the request handler that handles (i.e.
-  the standard or dismax SearchHandler) queries such that a separate request is
-  not needed to get suggestions.
-
-  IN OTHER WORDS, THERE IS REALLY GOOD CHANCE THE SETUP BELOW IS NOT WHAT YOU
-  WANT FOR YOUR PRODUCTION SYSTEM!
-  #############################################################################
-  -->
-  <requestHandler name="/spell" class="solr.SearchHandler" lazy="true">
-    <lst name="defaults">
-      <!-- omp = Only More Popular -->
-      <str name="spellcheck.onlyMorePopular">false</str>
-      <!-- exr = Extended Results -->
-      <str name="spellcheck.extendedResults">false</str>
-      <!--  The number of suggestions to return -->
-      <str name="spellcheck.count">1</str>
-    </lst>
-    <arr name="last-components">
-      <str>spellcheck</str>
-    </arr>
-  </requestHandler>
-
-  <searchComponent name="tvComponent" class="org.apache.solr.handler.component.TermVectorComponent"/>
-  <!-- A Req Handler for working with the tvComponent.  This is purely as an example.
-  You will likely want to add the component to your already specified request handlers. -->
-  <requestHandler name="tvrh" class="org.apache.solr.handler.component.SearchHandler">
-    <lst name="defaults">
-      <bool name="tv">true</bool>
-    </lst>
-    <arr name="last-components">
-      <str>tvComponent</str>
-    </arr>
-  </requestHandler>
-
-  <!-- Clustering Component
-       http://wiki.apache.org/solr/ClusteringComponent
-       This relies on third party jars which are not included in the release.
-       To use this component (and the "/clustering" handler)
-       Those jars will need to be downloaded, and you'll need to set the
-       solr.cluster.enabled system property when running solr...
-          java -Dsolr.clustering.enabled=true -jar start.jar
-    -->
-  <searchComponent
-    name="clusteringComponent"
-    enable="${solr.clustering.enabled:false}"
-    class="org.apache.solr.handler.clustering.ClusteringComponent" >
-    <!-- Declare an engine -->
-    <lst name="engine">
-      <!-- The name, only one can be named "default" -->
-      <str name="name">default</str>
-      <!-- 
-           Class name of Carrot2 clustering algorithm. Currently available algorithms are:
-           
-           * org.carrot2.clustering.lingo.LingoClusteringAlgorithm
-           * org.carrot2.clustering.stc.STCClusteringAlgorithm
-           
-           See http://project.carrot2.org/algorithms.html for the algorithm's characteristics.
-        -->
-      <str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
-      <!-- 
-           Overriding values for Carrot2 default algorithm attributes. For a description
-           of all available attributes, see: http://download.carrot2.org/stable/manual/#chapter.components.
-           Use attribute key as name attribute of str elements below. These can be further
-           overridden for individual requests by specifying attribute key as request
-           parameter name and attribute value as parameter value.
-        -->
-      <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
-    </lst>
-    <lst name="engine">
-      <str name="name">stc</str>
-      <str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
-    </lst>
-  </searchComponent>
-  <requestHandler name="/clustering"
-                  enable="${solr.clustering.enabled:false}"
-                  class="solr.SearchHandler">
-     <lst name="defaults">
-       <bool name="clustering">true</bool>
-       <str name="clustering.engine">default</str>
-       <bool name="clustering.results">true</bool>
-       <!-- The title field -->
-       <str name="carrot.title">name</str>
-       <str name="carrot.url">id</str>
-       <!-- The field to cluster on -->
-       <str name="carrot.snippet">features</str>
-       <!-- produce summaries -->
-       <bool name="carrot.produceSummary">true</bool>
-       <!-- the maximum number of labels per cluster -->
-       <!--<int name="carrot.numDescriptions">5</int>-->
-       <!-- produce sub clusters -->
-       <bool name="carrot.outputSubClusters">false</bool>
-    </lst>     
-    <arr name="last-components">
-      <str>clusteringComponent</str>
-    </arr>
-  </requestHandler>
-  
-  <!-- Solr Cell: http://wiki.apache.org/solr/ExtractingRequestHandler -->
-  <requestHandler name="/update/extract" class="org.apache.solr.handler.extraction.ExtractingRequestHandler" startup="lazy">
-    <lst name="defaults">
-      <!-- All the main content goes into "text"... if you need to return
-           the extracted text or do highlighting, use a stored field. -->
-      <str name="fmap.content">text</str>
-      <str name="lowernames">true</str>
-      <str name="uprefix">ignored_</str>
-
-      <!-- capture link hrefs but ignore div attributes -->
-      <str name="captureAttr">true</str>
-      <str name="fmap.a">links</str>
-      <str name="fmap.div">ignored_</str>
-    </lst>
-  </requestHandler>
-
-
-  <!-- A component to return terms and document frequency of those terms.
-       This component does not yet support distributed search. -->
-  <searchComponent name="termsComponent" class="org.apache.solr.handler.component.TermsComponent"/>
-
-  <requestHandler name="/terms" class="org.apache.solr.handler.component.SearchHandler">
-     <lst name="defaults">
-      <bool name="terms">true</bool>
-    </lst>     
-    <arr name="components">
-      <str>termsComponent</str>
-    </arr>
-  </requestHandler>
-
-
-  <!-- a search component that enables you to configure the top results for
-       a given query regardless of the normal lucene scoring.-->
-  <searchComponent name="elevator" class="solr.QueryElevationComponent" >
-    <!-- pick a fieldType to analyze queries -->
-    <str name="queryFieldType">string</str>
-    <str name="config-file">elevate.xml</str>
-  </searchComponent>
-
-  <!-- a request handler utilizing the elevator component -->
-  <requestHandler name="/elevate" class="solr.SearchHandler" startup="lazy">
-    <lst name="defaults">
-      <str name="echoParams">explicit</str>
-    </lst>
-    <arr name="last-components">
-      <str>elevator</str>
-    </arr>
-  </requestHandler>
-
-
-  <!-- Update request handler.  
-
-       Note: Since solr1.1 requestHandlers requires a valid content type header if posted in
-       the body. For example, curl now requires: -H 'Content-type:text/xml; charset=utf-8'
-       The response format differs from solr1.1 formatting and returns a standard error code.
-       To enable solr1.1 behavior, remove the /update handler or change its path
-    -->
-  <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
-
-
-  <requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />
-
-  <!--
-   Analysis request handler.  Since Solr 1.3.  Use to return how a document is analyzed.  Useful
-   for debugging and as a token server for other types of applications.
-
-   This is deprecated in favor of the improved DocumentAnalysisRequestHandler and FieldAnalysisRequestHandler
-
-   <requestHandler name="/analysis" class="solr.AnalysisRequestHandler" />
-   -->
-
-  <!--
-    An analysis handler that provides a breakdown of the analysis process of provided docuemnts. This handler expects a
-    (single) content stream with the following format:
-
-    <docs>
-      <doc>
-        <field name="id">1</field>
-        <field name="name">The Name</field>
-        <field name="text">The Text Value</field>
-      <doc>
-      <doc>...</doc>
-      <doc>...</doc>
-      ...
-    </docs>
-
-    Note: Each document must contain a field which serves as the unique key. This key is used in the returned
-    response to assoicate an analysis breakdown to the analyzed document.
-
-    Like the FieldAnalysisRequestHandler, this handler also supports query analysis by
-    sending either an "analysis.query" or "q" request paraemter that holds the query text to be analyized. It also
-    supports the "analysis.showmatch" parameter which when set to true, all field tokens that match the query
-    tokens will be marked as a "match".
-  -->
-  <requestHandler name="/analysis/document" class="solr.DocumentAnalysisRequestHandler" />
-
-  <!--
-    RequestHandler that provides much the same functionality as analysis.jsp. Provides the ability
-    to specify multiple field types and field names in the same request and outputs index-time and
-    query-time analysis for each of them.
-
-    Request parameters are:
-    analysis.fieldname - The field name whose analyzers are to be used
-    analysis.fieldtype - The field type whose analyzers are to be used
-    analysis.fieldvalue - The text for index-time analysis
-    q (or analysis.q) - The text for query time analysis
-    analysis.showmatch (true|false) - When set to true and when query analysis is performed, the produced
-                                      tokens of the field value analysis will be marked as "matched" for every
-                                      token that is produces by the query analysis
-   -->
-  <requestHandler name="/analysis/field" class="solr.FieldAnalysisRequestHandler" />
-
-
-  <!-- CSV update handler, loaded on demand -->
-  <requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
-
-
-  <!-- 
-   Admin Handlers - This will register all the standard admin RequestHandlers.  Adding 
-   this single handler is equivalent to registering:
-   
-  <requestHandler name="/admin/luke"       class="org.apache.solr.handler.admin.LukeRequestHandler" />
-  <requestHandler name="/admin/system"     class="org.apache.solr.handler.admin.SystemInfoHandler" />
-  <requestHandler name="/admin/plugins"    class="org.apache.solr.handler.admin.PluginInfoHandler" />
-  <requestHandler name="/admin/threads"    class="org.apache.solr.handler.admin.ThreadDumpHandler" />
-  <requestHandler name="/admin/properties" class="org.apache.solr.handler.admin.PropertiesRequestHandler" />
-  <requestHandler name="/admin/file"       class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
-  
-  If you wish to hide files under ${solr.home}/conf, explicitly register the ShowFileRequestHandler using:
-  <requestHandler name="/admin/file" class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
-    <lst name="invariants">
-     <str name="hidden">synonyms.txt</str> 
-     <str name="hidden">anotherfile.txt</str> 
-    </lst>
-  </requestHandler>
-  -->
-  <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
-
-  <!-- ping/healthcheck -->
-  <requestHandler name="/admin/ping" class="PingRequestHandler">
-    <lst name="defaults">
-      <str name="qt">standard</str>
-      <str name="q">solrpingquery</str>
-      <str name="echoParams">all</str>
-    </lst>
-  </requestHandler>
-
-  <!-- Echo the request contents back to the client -->
-  <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
-    <lst name="defaults">
-     <str name="echoParams">explicit</str> <!-- for all params (including the default etc) use: 'all' -->
-     <str name="echoHandler">true</str>
-    </lst>
-  </requestHandler>
-
-  <highlighting>
-   <!-- Configure the standard fragmenter -->
-   <!-- This could most likely be commented out in the "default" case -->
-   <fragmenter name="gap" class="org.apache.solr.highlight.GapFragmenter" default="true">
-    <lst name="defaults">
-     <int name="hl.fragsize">100</int>
-    </lst>
-   </fragmenter>
-
-   <!-- A regular-expression-based fragmenter (f.i., for sentence extraction) -->
-   <fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter">
-    <lst name="defaults">
-      <!-- slightly smaller fragsizes work better because of slop -->
-      <int name="hl.fragsize">70</int>
-      <!-- allow 50% slop on fragment sizes -->
-      <float name="hl.regex.slop">0.5</float>
-      <!-- a basic sentence pattern -->
-      <str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
-    </lst>
-   </fragmenter>
-
-   <!-- Configure the standard formatter -->
-   <formatter name="html" class="org.apache.solr.highlight.HtmlFormatter" default="true">
-    <lst name="defaults">
-     <str name="hl.simple.pre"><![CDATA[<em>]]></str>
-     <str name="hl.simple.post"><![CDATA[</em>]]></str>
-    </lst>
-   </formatter>
-  </highlighting>
-
-  <!-- An example dedup update processor that creates the "id" field on the fly
-       based on the hash code of some other fields.  This example has overwriteDupes
-       set to false since we are using the id field as the signatureField and Solr
-       will maintain uniqueness based on that anyway. 
-       
-       You have to link the chain to an update handler above to use it ie:
-         <requestHandler name="/update "class="solr.XmlUpdateRequestHandler">
-           <lst name="defaults">
-             <str name="update.processor">dedupe</str>
-           </lst>
-         </requestHandler>  
-  -->
-  <!--
-  <updateRequestProcessorChain name="dedupe">
-    <processor class="org.apache.solr.update.processor.SignatureUpdateProcessorFactory">
-      <bool name="enabled">true</bool>
-      <str name="signatureField">id</str>
-      <bool name="overwriteDupes">false</bool>
-      <str name="fields">name,features,cat</str>
-      <str name="signatureClass">org.apache.solr.update.processor.Lookup3Signature</str>
-    </processor>
-    <processor class="solr.LogUpdateProcessorFactory" />
-    <processor class="solr.RunUpdateProcessorFactory" />
-  </updateRequestProcessorChain>
-  -->
-
-
-  <!-- queryResponseWriter plugins... query responses will be written using the
-    writer specified by the 'wt' request parameter matching the name of a registered
-    writer.
-    The "default" writer is the default and will be used if 'wt' is not specified 
-    in the request. XMLResponseWriter will be used if nothing is specified here.
-    The json, python, and ruby writers are also available by default.
-
-    <queryResponseWriter name="xml" class="org.apache.solr.request.XMLResponseWriter" default="true"/>
-    <queryResponseWriter name="json" class="org.apache.solr.request.JSONResponseWriter"/>
-    <queryResponseWriter name="python" class="org.apache.solr.request.PythonResponseWriter"/>
-    <queryResponseWriter name="ruby" class="org.apache.solr.request.RubyResponseWriter"/>
-    <queryResponseWriter name="php" class="org.apache.solr.request.PHPResponseWriter"/>
-    <queryResponseWriter name="phps" class="org.apache.solr.request.PHPSerializedResponseWriter"/>
-
-    <queryResponseWriter name="custom" class="com.example.MyResponseWriter"/>
-  -->
-
-  <!-- XSLT response writer transforms the XML output by any xslt file found
-       in Solr's conf/xslt directory.  Changes to xslt files are checked for
-       every xsltCacheLifetimeSeconds.  
-   -->
-  <queryResponseWriter name="xslt" class="org.apache.solr.request.XSLTResponseWriter">
-    <int name="xsltCacheLifetimeSeconds">5</int>
-  </queryResponseWriter>
-
-
-  <!-- example of registering a query parser
-  <queryParser name="lucene" class="org.apache.solr.search.LuceneQParserPlugin"/>
-  -->
-
-  <!-- example of registering a custom function parser 
-  <valueSourceParser name="myfunc" class="com.mycompany.MyValueSourceParser" />
-  -->
-
-  <!-- config for the admin interface -->
-  <admin>
-    <defaultQuery>solr</defaultQuery>
-
-    <!-- configure a healthcheck file for servers behind a loadbalancer
-    <healthcheck type="file">server-enabled</healthcheck>
-    -->
-  </admin>
-
-  <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
-    <lst name="defaults">
-      <str name="config">/etc/solr/conf/data-config.xml</str>
-    </lst>
-  </requestHandler>
-
-
-</config>
...	...	@@ -0,0 +1,5 @@
	1	+include colab/templates/*.html
	2	+include colab/super_archives/templates/*.html
	3	+include colab/super_archives/fixtures/initial_data.json
	4	+recursive-include etc *
	5	+recursive-include colab/static *
0	6	\ No newline at end of file
...	...
...	...	@@ -0,0 +1,106 @@
	1	+Installation instructions for Ubuntu 10.04
	2	+-------------------------------------------
	3	+
	4	+* Install Apache2 with WSGI support:
	5	+
	6	+ * apt-get install apache2 libapache2-mod-wsgi
	7	+
	8	+* Install dependencies to compile psycopg2:
	9	+
	10	+ * apt-get build-dep python-psycopg2
	11	+
	12	+* Install Python PIP and update it:
	13	+
	14	+ * apt-get install python-pip
	15	+ * pip install -U pip
	16	+
	17	+* Install python virtualenv:
	18	+
	19	+ * pip install virtualenv
	20	+
	21	+* Create a virtualenv for the deploy
	22	+
	23	+ * mkdir /usr/local/django/
	24	+ * virtualenv /usr/local/django/colab/
	25	+
	26	+* Download the colab src code:
	27	+
	28	+ * hg clone https://bitbucket.org/seocam/atu-colab /usr/local/src/colab/
	29	+
	30	+* Install the django site:
	31	+
	32	+ * pip install /usr/local/src/colab -E /usr/local/django/colab/
	33	+
	34	+* Configure your database settings in /usr/local/django/colab/lib/python2.6/site-packages/settings_local.py
	35	+
	36	+* Enable the colab site on apache and reload it:
	37	+
	38	+ * ln -s /usr/local/django/colab/apache-site/colab /etc/apache2/sites-available
	39	+ * a2ensite colab
	40	+ * service apache2 restart
	41	+
	42	+
	43	+Configuring server to send emails
	44	+----------------------------------
	45	+
	46	+* Install postfix and mailutils:
	47	+
	48	+ * apt-get install mailutils postfix
	49	+
	50	+* Update the file /etc/aliases adding users that should receive root's messages and run the update command:
	51	+
	52	+ * newaliases
	53	+
	54	+
	55	+Cron job to import emails
	56	+---------------------------
	57	+
	58	+* Install sshfs:
	59	+
	60	+ * apt-get install sshfs autofs
	61	+
	62	+* Create SSH keys. You should use a password but this tutorial won't cover it (if you use you will need to install and configure keychain process to be able to proceed):
	63	+
	64	+ * ssh-keygen
	65	+
	66	+* Copy the content of your key (/root/.ssh/id_rsa.pub) to the file /root/.ssh/authorized_keys on the mailinglist server.
	67	+
	68	+* Append the following content to /etc/auto.master file:
	69	+
	70	+ * /usr/local/django/colab/mnt /usr/local/django/colab/autofs/listas --timeout=600,--ghost
	71	+
	72	+* Restart autofs:
	73	+
	74	+ * service autofs restart
	75	+
	76	+* Link cron script into /etc/cron.d/ folder:
	77	+
	78	+ * ln -s /usr/local/django/colab/cron.d/colab_import_emails /etc/cron.d/
	79	+
	80	+* From now on the emails should be imported every minute
	81	+
	82	+
	83	+Cron job to reindex Solr
	84	+-------------------------
	85	+
	86	+* Install wget:
	87	+
	88	+ * apt-get install wget
	89	+
	90	+* Link cron script into /etc/cron.d/ folder:
	91	+
	92	+ * ln -s /usr/local/django/colab/cron.d/colab_solr_reindex /etc/cron.d/
	93	+
	94	+* From now on delta reindex should run every 10 minutes and full reindex once a day.
	95	+
	96	+
	97	+Updating an installed version
	98	+------------------------------
	99	+
	100	+* Update the source code:
	101	+
	102	+ * cd /usr/local/src/colab/
	103	+ * hg pull
	104	+ * hg up
	105	+ * pip install /usr/local/src/colab/ -E /usr/local/django/colab/ -U
	106	+ * service apache2 restart
0	107	\ No newline at end of file
...	...
...	...	@@ -0,0 +1,64 @@
	1	+TODO
	2	+-----
	3	+
	4	+* Sergio: Permitir que usuario atualize nome e sobrenome
	5	+
	6	+* Jean: Pagina Reporte um problema
	7	+* Jean: Pagina Contribua
	8	+
	9	+* Yure: Adicionar data do ultimo import de emails no footer
	10	+* Yure: Detectar links no conteudo e exibi-los como tal
	11	+* Yure: BUG: Display of HTML emails are wrong
	12	+* Yure: Cadastrar usuário em lista pelo formulario de cadastro
	13	+* Arquivo "search.html" existente em "atu-colab/colab/templates" pode ser melhorado com relação ao conteúdo repetitivo dos "Tipos" exibidos no "Filtro" da página
	14	+* Criar validador de urls para twitter, facebook e página pessoal do user profile
	15	+
	16	+* Configurar ADMINS no arquivo settings_local.py
	17	+* HTTPS para o trac, subversion e colab
	18	+* Autorizar usuarios a commitar no svn pelo django.contrib.admin
	19	+* Enviar emails para usuarios pedindo que se cadastrem no novo colab
	20	+* Timezones no trac/colab/solr nao estao compativeis
	21	+
	22	+* Template de login nao exibe corretamente no firefox/linux
	23	+* Quando usuario se cadastra com email errado o email nunca eh validado,
	24	+e o username fica preso 'pra sempre'.
	25	+* Nome dos usuarios errado nos emails que vem do Solr
	26	+* Adicionar ordering na busca
	27	+* Criar tipo usuario no solr
	28	+* Substituir sistema de cadastro por django-registration
	29	+* Utilizar pysolr para efetuar queries no Solr
	30	+* Melhorar buscas (case insensitive match, palavras com acentos)
	31	+* Indexar nome do repositorio como campo e exibi-lo no titulo dos changesets retornados
	32	+* Criacao de repositorios distribuidos pela interface do colab
	33	+* Link para última msg recebida na thread
	34	+* Fazer thread querysets ter um objeto (most_relevant_message)
	35	+* Implementar enviar email
	36	+* BUG: alguns subjects comecam e terminam com [] fazendo com que a RE de limpeza apague todo o subject.
	37	+* BUG: mensagens importadas por listas erradas
	38	+* Implementar login/signup usando LDAP
	39	+* Claime email address
	40	+* Merge emails dos usuarios
	41	+* Implementar badge system
	42	+* Melhorar filtros
	43	+* Link do thread preview deve enviar para mensagem da thread (anchor) (Útil? discutir com Jean)
	44	+* Tornar todas as strings traduziveis
	45	+* Sugestão de como a divisão do edital seria melhor
	46	+* Indexar anexos da wiki (using Tika http://wiki.apache.org/solr/ExtractingRequestHandler)
	47	+* Filtrar usando calendario (como google analytics)
	48	+* Melhorar relevancia das buscas usando dismax queryparser
	49	+* Chat estilo Gmail
	50	+* Sistema de gerencia de conteúdo
	51	+* Versao mobile
	52	+* Exibir discussões relacionadas na barra da direita das discussões
	53	+* Migração e reorganização do conteúdo do trac/wiki para o novo colab
	54	+* Ao importar mensagem sem subject enviar email avisando o usuario que ele esta enviando um email sem o campos "Assunto"
	55	+* Remover ou ocultar trechos da mensagem que iniciem com ">" assim como o Gmail.
	56	+* Contar page views no trac (ticket, wiki e changeset) e utiliza-los para rankear paginas nas buscas
	57	+* Mostrar highlight nas buscas
	58	+* Sistema de tags para as mensagens
	59	+* Pagina home para cada lista com os mesmo filtros da home atual
	60	+* Permitir que usuario entre e saia de listas ao editar perfil
	61	+* Planet Interlegis (agregador de blogs)
	62	+* Filtros especificos para tipos diferentes na busca
	63	+* Indice criado manualmente. Automatizar:
	64	+ * create index super_archives_message_body_idx ON super_archives_message ((substring(body,0,1024)));
...	...
1	1
	2	+from django.core.cache import cache
2	3	from django.db import IntegrityError
3	4	from django.core.exceptions import ObjectDoesNotExist
4	5	from django.contrib.auth.decorators import login_required
...	...	@@ -6,7 +7,7 @@ from django.contrib.auth.decorators import login_required
6	7	from piston.utils import rc
7	8	from piston.handler import BaseHandler
8	9
9		-from super_archives.models import Message
	10	+from colab.super_archives.models import Message, PageHit
10	11
11	12
12	13	class VoteHandler(BaseHandler):
...	...	@@ -38,5 +39,33 @@ class VoteHandler(BaseHandler):
38	39	return rc.DELETED
39	40
40	41
	42	+class CountHandler(BaseHandler):
	43	+ allowed_methods = ('POST')
	44	+
	45	+ def create(self, request):
	46	+ """Add one page view for the given url"""
	47	+
	48	+ # If missing the path_info argument we can't do anything
	49	+ path_info = request.POST.get('path_info')
	50	+ if not path_info:
	51	+ return rc.BAD_REQUEST
	52	+
	53	+ # Here we cache the user's IP to ensure that the same
	54	+ # IP won't hit the same page again for while
	55	+ ip_addr = request.META.get('REMOTE_ADDR')
	56	+ page_hits_cache = cache.get('page_hits', {})
	57	+ duplicate = page_hits_cache.get(path_info, {}).get(ip_addr)
41	58
	59	+ if duplicate:
	60	+ return rc.DUPLICATE_ENTRY
	61	+ else:
	62	+ page_hits_cache.update({path_info: {ip_addr: True }})
	63	+ cache.set('page_hits', page_hits_cache)
	64	+
	65	+ # Everything ok, so just increment the page count
	66	+ page_hit = PageHit.objects.get_or_create(url_path=path_info)[0]
	67	+ page_hit.hit_count += 1
	68	+ page_hit.save()
	69	+
	70	+ return rc.CREATED
42	71
...	...
...	...	@@ -2,11 +2,13 @@ from django.conf.urls.defaults import patterns, include, url
2	2
3	3	from piston.resource import Resource
4	4
5		-from api.handlers import VoteHandler
	5	+from colab.api.handlers import VoteHandler, CountHandler
6	6
7	7
8	8	vote_handler = Resource(VoteHandler)
	9	+count_handler = Resource(CountHandler)
9	10
10	11	urlpatterns = patterns('',
11	12	url(r'message/(?P<message_id>\d+)/vote$', vote_handler),
	13	+ url(r'hit/$', count_handler),
12	14	)
13	15	\ No newline at end of file
...	...
...	...	@@ -1,27 +0,0 @@
1		-import sys
2		-import site
3		-import os
4		-
5		-app_path = os.path.abspath(os.path.dirname(__file__))
6		-app_parent_path = os.path.abspath(os.path.join(app_path, os.path.pardir))
7		-
8		-vepath = '/home/seocam/.virtualenvs/colab/lib/python2.6/site-packages'
9		-
10		-prev_sys_path = list(sys.path)
11		-# add the site-packages of our virtualenv as a site dir
12		-site.addsitedir(vepath)
13		-# add the app's directory to the PYTHONPATH
14		-sys.path.append(app_path)
15		-sys.path.append(app_parent_path)
16		-
17		-# reorder sys.path so new directories from the addsitedir show up first
18		-new_sys_path = [p for p in sys.path if p not in prev_sys_path]
19		-for item in new_sys_path:
20		- sys.path.remove(item)
21		-sys.path[:0] = new_sys_path
22		-
23		-# import from down here to pull in possible virtualenv django install
24		-from django.core.handlers.wsgi import WSGIHandler
25		-os.environ['DJANGO_SETTINGS_MODULE'] = 'colab.settings'
26		-application = WSGIHandler()
27		-
...	...	@@ -5,8 +5,12 @@ import os.path
5	5	DEBUG = True
6	6	TEMPLATE_DEBUG = DEBUG
7	7
	8	+if DEBUG:
	9	+ import logging
	10	+ logging.root.setLevel(logging.DEBUG)
	11	+
	12	+
8	13	ADMINS = (
9		- # ('Your Name', 'your_email@example.com'),
10	14	)
11	15
12	16	PROJECT_PATH = os.path.abspath(os.path.dirname(__file__))
...	...	@@ -22,7 +26,7 @@ LOGIN_URL = '/login/'
22	26	# timezone as the operating system.
23	27	# If running in a Windows environment this must be set to the same as your
24	28	# system time zone.
25		-TIME_ZONE = 'America/Chicago'
	29	+TIME_ZONE = 'America/Sao_Paulo'
26	30
27	31	# Language code for this installation. All choices can be found here:
28	32	# http://www.i18nguy.com/unicode/language-identifiers.html
...	...	@@ -39,12 +43,12 @@ USE_I18N = True
39	43	USE_L10N = True
40	44
41	45	MEDIA_ROOT = os.path.join(PROJECT_PATH, os.pardir, 'site_media/media')
42		-MEDIA_URL = '/site_media/media/'
	46	+MEDIA_URL = '/media/'
43	47
44	48	STATIC_ROOT = os.path.join(PROJECT_PATH, os.pardir, 'site_media/static')
45		-STATIC_URL = '/site_media/static/'
	49	+STATIC_URL = '/static/'
46	50
47		-#ADMIN_MEDIA_PREFIX = '/site_media/static/admin/'
	51	+ADMIN_MEDIA_PREFIX = '/static/admin/'
48	52
49	53	# Additional locations of static files
50	54	STATICFILES_DIRS = (
...	...	@@ -66,12 +70,23 @@ TEMPLATE_LOADERS = (
66	70	# 'django.template.loaders.eggs.Loader',
67	71	)
68	72
	73	+TEMPLATE_CONTEXT_PROCESSORS = (
	74	+ 'django.contrib.auth.context_processors.auth',
	75	+ 'django.core.context_processors.debug',
	76	+ 'django.core.context_processors.i18n',
	77	+ 'django.core.context_processors.media',
	78	+ 'django.core.context_processors.static',
	79	+ 'django.contrib.messages.context_processors.messages',
	80	+ 'django.core.context_processors.request',
	81	+)
	82	+
69	83	MIDDLEWARE_CLASSES = (
70	84	'django.middleware.common.CommonMiddleware',
71	85	'django.contrib.sessions.middleware.SessionMiddleware',
72	86	'django.middleware.csrf.CsrfViewMiddleware',
73	87	'django.contrib.auth.middleware.AuthenticationMiddleware',
74	88	'django.contrib.messages.middleware.MessageMiddleware',
	89	+ #'debug_toolbar.middleware.DebugToolbarMiddleware',
75	90	)
76	91
77	92	ROOT_URLCONF = 'colab.urls'
...	...	@@ -91,17 +106,18 @@ INSTALLED_APPS = (
91	106	'django.contrib.messages',
92	107	'django.contrib.staticfiles',
93	108	# Uncomment the next line to enable the admin:
94		- # 'django.contrib.admin',
	109	+ 'django.contrib.admin',
95	110	# Uncomment the next line to enable admin documentation:
96	111	# 'django.contrib.admindocs',
97	112
98	113	# Not standard apps
99	114	'south',
100		- 'registration',
	115	+ 'cliauth',
	116	+ #'debug_toolbar',
101	117
102	118	# My apps
103		- 'super_archives',
104		- 'api',
	119	+ 'colab.super_archives',
	120	+ 'colab.api',
105	121	)
106	122
107	123	# A sample logging configuration. The only tangible logging
...	...	@@ -115,7 +131,8 @@ LOGGING = {
115	131	'handlers': {
116	132	'mail_admins': {
117	133	'level': 'ERROR',
118		- 'class': 'django.utils.log.AdminEmailHandler'
	134	+ 'class': 'django.utils.log.AdminEmailHandler',
	135	+ 'include_html': True,
119	136	}
120	137	},
121	138	'loggers': {
...	...	@@ -127,5 +144,18 @@ LOGGING = {
127	144	}
128	145	}
129	146
	147	+SERVER_EMAIL = '"Colab Interlegis" <noreply@interlegis.leg.br>'
	148	+EMAIL_HOST_USER = SERVER_EMAIL
	149	+
	150	+#SOLR_HOSTNAME = 'solr.interlegis.leg.br'
	151	+SOLR_HOSTNAME = '10.1.2.154'
	152	+SOLR_PORT = '8080'
	153	+SOLR_SELECT_PATH = '/solr/select'
	154	+
	155	+SOLR_COLAB_URI = 'http://colab.interlegis.gov.br'
	156	+SOLR_BASE_QUERY = """
	157	+ (Type:changeset OR Type:ticket OR Type:wiki OR Type:thread)
	158	+"""
	159	+
130	160	from settings_local import *
131	161
...	...
...	...	@@ -0,0 +1,18 @@
	1	+
	2	+DEBUG = True
	3	+TEMPLATE_DEBUG = DEBUG
	4	+
	5	+ADMINS = (
	6	+)
	7	+
	8	+MANAGERS = ADMINS
	9	+
	10	+DATABASES = {
	11	+ 'default': {
	12	+ 'ENGINE': 'django.db.backends.sqlite3',
	13	+ 'NAME': 'colab.db',
	14	+ }
	15	+}
	16	+
	17	+# Make this unique, and don't share it with anybody.
	18	+SECRET_KEY = ')(jksdfhsjkadfhjkh234ns!8fqu-1186h$vuj'
...	...
...	...	@@ -0,0 +1,50 @@
	1	+#!/usr/bin/env python
	2	+# encoding: utf-8
	3	+
	4	+from django.conf import settings
	5	+from django.utils.html import strip_tags
	6	+from django.utils.translation import ugettext as _
	7	+from django.core.mail import EmailMultiAlternatives
	8	+from django.template.loader import render_to_string
	9	+
	10	+
	11	+def send_verification_email(request, user):
	12	+
	13	+ subject = _(u'Colab: Verificação de email')
	14	+ from_ = settings.SERVER_EMAIL
	15	+ to = user.email
	16	+
	17	+ email_data = {
	18	+ 'hash': user.profile.verification_hash,
	19	+ 'server_name': request.get_host(),
	20	+ }
	21	+
	22	+ html_content = render_to_string('email_signup-email-confirmation.html',
	23	+ email_data)
	24	+ text_content = strip_tags(html_content)
	25	+ email_msg = EmailMultiAlternatives(subject, text_content, from_, [to])
	26	+ email_msg.attach_alternative(html_content, 'text/html')
	27	+ email_msg.send()
	28	+
	29	+
	30	+def send_reset_password_email(request, user):
	31	+
	32	+ subject = _(u'Altereção de senha do Colab Interlegis')
	33	+ from_ = settings.SERVER_EMAIL
	34	+ to = user.email
	35	+
	36	+ email_data = {
	37	+ 'hash': user.profile.verification_hash,
	38	+ 'server_name': request.get_host(),
	39	+ 'username': user.username,
	40	+ }
	41	+
	42	+ html_content = render_to_string('email_account-reset-password.html',
	43	+ email_data)
	44	+ text_content = strip_tags(html_content)
	45	+
	46	+ email_msg = EmailMultiAlternatives(subject, text_content, from_, [to])
	47	+ email_msg.attach_alternative(html_content, 'text/html')
	48	+ email_msg.send()
	49	+
	50	+
0	51	\ No newline at end of file
...	...
...	...	@@ -0,0 +1,387 @@
	1	+"""SocksiPy - Python SOCKS module.
	2	+Version 1.00
	3	+
	4	+Copyright 2006 Dan-Haim. All rights reserved.
	5	+
	6	+Redistribution and use in source and binary forms, with or without modification,
	7	+are permitted provided that the following conditions are met:
	8	+1. Redistributions of source code must retain the above copyright notice, this
	9	+ list of conditions and the following disclaimer.
	10	+2. Redistributions in binary form must reproduce the above copyright notice,
	11	+ this list of conditions and the following disclaimer in the documentation
	12	+ and/or other materials provided with the distribution.
	13	+3. Neither the name of Dan Haim nor the names of his contributors may be used
	14	+ to endorse or promote products derived from this software without specific
	15	+ prior written permission.
	16	+
	17	+THIS SOFTWARE IS PROVIDED BY DAN HAIM "AS IS" AND ANY EXPRESS OR IMPLIED
	18	+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
	19	+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
	20	+EVENT SHALL DAN HAIM OR HIS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
	21	+INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
	22	+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA
	23	+OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
	24	+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
	25	+OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMANGE.
	26	+
	27	+
	28	+This module provides a standard socket-like interface for Python
	29	+for tunneling connections through SOCKS proxies.
	30	+
	31	+"""
	32	+
	33	+import socket
	34	+import struct
	35	+
	36	+PROXY_TYPE_SOCKS4 = 1
	37	+PROXY_TYPE_SOCKS5 = 2
	38	+PROXY_TYPE_HTTP = 3
	39	+
	40	+_defaultproxy = None
	41	+_orgsocket = socket.socket
	42	+
	43	+class ProxyError(Exception):
	44	+ def __init__(self, value):
	45	+ self.value = value
	46	+ def __str__(self):
	47	+ return repr(self.value)
	48	+
	49	+class GeneralProxyError(ProxyError):
	50	+ def __init__(self, value):
	51	+ self.value = value
	52	+ def __str__(self):
	53	+ return repr(self.value)
	54	+
	55	+class Socks5AuthError(ProxyError):
	56	+ def __init__(self, value):
	57	+ self.value = value
	58	+ def __str__(self):
	59	+ return repr(self.value)
	60	+
	61	+class Socks5Error(ProxyError):
	62	+ def __init__(self, value):
	63	+ self.value = value
	64	+ def __str__(self):
	65	+ return repr(self.value)
	66	+
	67	+class Socks4Error(ProxyError):
	68	+ def __init__(self, value):
	69	+ self.value = value
	70	+ def __str__(self):
	71	+ return repr(self.value)
	72	+
	73	+class HTTPError(ProxyError):
	74	+ def __init__(self, value):
	75	+ self.value = value
	76	+ def __str__(self):
	77	+ return repr(self.value)
	78	+
	79	+_generalerrors = ("success",
	80	+ "invalid data",
	81	+ "not connected",
	82	+ "not available",
	83	+ "bad proxy type",
	84	+ "bad input")
	85	+
	86	+_socks5errors = ("succeeded",
	87	+ "general SOCKS server failure",
	88	+ "connection not allowed by ruleset",
	89	+ "Network unreachable",
	90	+ "Host unreachable",
	91	+ "Connection refused",
	92	+ "TTL expired",
	93	+ "Command not supported",
	94	+ "Address type not supported",
	95	+ "Unknown error")
	96	+
	97	+_socks5autherrors = ("succeeded",
	98	+ "authentication is required",
	99	+ "all offered authentication methods were rejected",
	100	+ "unknown username or invalid password",
	101	+ "unknown error")
	102	+
	103	+_socks4errors = ("request granted",
	104	+ "request rejected or failed",
	105	+ "request rejected because SOCKS server cannot connect to identd on the client",
	106	+ "request rejected because the client program and identd report different user-ids",
	107	+ "unknown error")
	108	+
	109	+def setdefaultproxy(proxytype=None,addr=None,port=None,rdns=True,username=None,password=None):
	110	+ """setdefaultproxy(proxytype, addr[, port[, rdns[, username[, password]]]])
	111	+ Sets a default proxy which all further socksocket objects will use,
	112	+ unless explicitly changed.
	113	+ """
	114	+ global _defaultproxy
	115	+ _defaultproxy = (proxytype,addr,port,rdns,username,password)
	116	+
	117	+class socksocket(socket.socket):
	118	+ """socksocket([family[, type[, proto]]]) -> socket object
	119	+
	120	+ Open a SOCKS enabled socket. The parameters are the same as
	121	+ those of the standard socket init. In order for SOCKS to work,
	122	+ you must specify family=AF_INET, type=SOCK_STREAM and proto=0.
	123	+ """
	124	+
	125	+ def __init__(self, family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0, _sock=None):
	126	+ _orgsocket.__init__(self,family,type,proto,_sock)
	127	+ if _defaultproxy != None:
	128	+ self.__proxy = _defaultproxy
	129	+ else:
	130	+ self.__proxy = (None, None, None, None, None, None)
	131	+ self.__proxysockname = None
	132	+ self.__proxypeername = None
	133	+
	134	+ def __recvall(self, bytes):
	135	+ """__recvall(bytes) -> data
	136	+ Receive EXACTLY the number of bytes requested from the socket.
	137	+ Blocks until the required number of bytes have been received.
	138	+ """
	139	+ data = ""
	140	+ while len(data) < bytes:
	141	+ data = data + self.recv(bytes-len(data))
	142	+ return data
	143	+
	144	+ def setproxy(self,proxytype=None,addr=None,port=None,rdns=True,username=None,password=None):
	145	+ """setproxy(proxytype, addr[, port[, rdns[, username[, password]]]])
	146	+ Sets the proxy to be used.
	147	+ proxytype - The type of the proxy to be used. Three types
	148	+ are supported: PROXY_TYPE_SOCKS4 (including socks4a),
	149	+ PROXY_TYPE_SOCKS5 and PROXY_TYPE_HTTP
	150	+ addr - The address of the server (IP or DNS).
	151	+ port - The port of the server. Defaults to 1080 for SOCKS
	152	+ servers and 8080 for HTTP proxy servers.
	153	+ rdns - Should DNS queries be preformed on the remote side
	154	+ (rather than the local side). The default is True.
	155	+ Note: This has no effect with SOCKS4 servers.
	156	+ username - Username to authenticate with to the server.
	157	+ The default is no authentication.
	158	+ password - Password to authenticate with to the server.
	159	+ Only relevant when username is also provided.
	160	+ """
	161	+ self.__proxy = (proxytype,addr,port,rdns,username,password)
	162	+
	163	+ def __negotiatesocks5(self,destaddr,destport):
	164	+ """__negotiatesocks5(self,destaddr,destport)
	165	+ Negotiates a connection through a SOCKS5 server.
	166	+ """
	167	+ # First we'll send the authentication packages we support.
	168	+ if (self.__proxy[4]!=None) and (self.__proxy[5]!=None):
	169	+ # The username/password details were supplied to the
	170	+ # setproxy method so we support the USERNAME/PASSWORD
	171	+ # authentication (in addition to the standard none).
	172	+ self.sendall("\x05\x02\x00\x02")
	173	+ else:
	174	+ # No username/password were entered, therefore we
	175	+ # only support connections with no authentication.
	176	+ self.sendall("\x05\x01\x00")
	177	+ # We'll receive the server's response to determine which
	178	+ # method was selected
	179	+ chosenauth = self.__recvall(2)
	180	+ if chosenauth[0] != "\x05":
	181	+ self.close()
	182	+ raise GeneralProxyError((1,_generalerrors[1]))
	183	+ # Check the chosen authentication method
	184	+ if chosenauth[1] == "\x00":
	185	+ # No authentication is required
	186	+ pass
	187	+ elif chosenauth[1] == "\x02":
	188	+ # Okay, we need to perform a basic username/password
	189	+ # authentication.
	190	+ self.sendall("\x01" + chr(len(self.__proxy[4])) + self.__proxy[4] + chr(len(self.proxy[5])) + self.__proxy[5])
	191	+ authstat = self.__recvall(2)
	192	+ if authstat[0] != "\x01":
	193	+ # Bad response
	194	+ self.close()
	195	+ raise GeneralProxyError((1,_generalerrors[1]))
	196	+ if authstat[1] != "\x00":
	197	+ # Authentication failed
	198	+ self.close()
	199	+ raise Socks5AuthError,((3,_socks5autherrors[3]))
	200	+ # Authentication succeeded
	201	+ else:
	202	+ # Reaching here is always bad
	203	+ self.close()
	204	+ if chosenauth[1] == "\xFF":
	205	+ raise Socks5AuthError((2,_socks5autherrors[2]))
	206	+ else:
	207	+ raise GeneralProxyError((1,_generalerrors[1]))
	208	+ # Now we can request the actual connection
	209	+ req = "\x05\x01\x00"
	210	+ # If the given destination address is an IP address, we'll
	211	+ # use the IPv4 address request even if remote resolving was specified.
	212	+ try:
	213	+ ipaddr = socket.inet_aton(destaddr)
	214	+ req = req + "\x01" + ipaddr
	215	+ except socket.error:
	216	+ # Well it's not an IP number, so it's probably a DNS name.
	217	+ if self.__proxy[3]==True:
	218	+ # Resolve remotely
	219	+ ipaddr = None
	220	+ req = req + "\x03" + chr(len(destaddr)) + destaddr
	221	+ else:
	222	+ # Resolve locally
	223	+ ipaddr = socket.inet_aton(socket.gethostbyname(destaddr))
	224	+ req = req + "\x01" + ipaddr
	225	+ req = req + struct.pack(">H",destport)
	226	+ self.sendall(req)
	227	+ # Get the response
	228	+ resp = self.__recvall(4)
	229	+ if resp[0] != "\x05":
	230	+ self.close()
	231	+ raise GeneralProxyError((1,_generalerrors[1]))
	232	+ elif resp[1] != "\x00":
	233	+ # Connection failed
	234	+ self.close()
	235	+ if ord(resp[1])<=8:
	236	+ raise Socks5Error(ord(resp[1]),_generalerrors[ord(resp[1])])
	237	+ else:
	238	+ raise Socks5Error(9,_generalerrors[9])
	239	+ # Get the bound address/port
	240	+ elif resp[3] == "\x01":
	241	+ boundaddr = self.__recvall(4)
	242	+ elif resp[3] == "\x03":
	243	+ resp = resp + self.recv(1)
	244	+ boundaddr = self.__recvall(resp[4])
	245	+ else:
	246	+ self.close()
	247	+ raise GeneralProxyError((1,_generalerrors[1]))
	248	+ boundport = struct.unpack(">H",self.__recvall(2))[0]
	249	+ self.__proxysockname = (boundaddr,boundport)
	250	+ if ipaddr != None:
	251	+ self.__proxypeername = (socket.inet_ntoa(ipaddr),destport)
	252	+ else:
	253	+ self.__proxypeername = (destaddr,destport)
	254	+
	255	+ def getproxysockname(self):
	256	+ """getsockname() -> address info
	257	+ Returns the bound IP address and port number at the proxy.
	258	+ """
	259	+ return self.__proxysockname
	260	+
	261	+ def getproxypeername(self):
	262	+ """getproxypeername() -> address info
	263	+ Returns the IP and port number of the proxy.
	264	+ """
	265	+ return _orgsocket.getpeername(self)
	266	+
	267	+ def getpeername(self):
	268	+ """getpeername() -> address info
	269	+ Returns the IP address and port number of the destination
	270	+ machine (note: getproxypeername returns the proxy)
	271	+ """
	272	+ return self.__proxypeername
	273	+
	274	+ def __negotiatesocks4(self,destaddr,destport):
	275	+ """__negotiatesocks4(self,destaddr,destport)
	276	+ Negotiates a connection through a SOCKS4 server.
	277	+ """
	278	+ # Check if the destination address provided is an IP address
	279	+ rmtrslv = False
	280	+ try:
	281	+ ipaddr = socket.inet_aton(destaddr)
	282	+ except socket.error:
	283	+ # It's a DNS name. Check where it should be resolved.
	284	+ if self.__proxy[3]==True:
	285	+ ipaddr = "\x00\x00\x00\x01"
	286	+ rmtrslv = True
	287	+ else:
	288	+ ipaddr = socket.inet_aton(socket.gethostbyname(destaddr))
	289	+ # Construct the request packet
	290	+ req = "\x04\x01" + struct.pack(">H",destport) + ipaddr
	291	+ # The username parameter is considered userid for SOCKS4
	292	+ if self.__proxy[4] != None:
	293	+ req = req + self.__proxy[4]
	294	+ req = req + "\x00"
	295	+ # DNS name if remote resolving is required
	296	+ # NOTE: This is actually an extension to the SOCKS4 protocol
	297	+ # called SOCKS4A and may not be supported in all cases.
	298	+ if rmtrslv==True:
	299	+ req = req + destaddr + "\x00"
	300	+ self.sendall(req)
	301	+ # Get the response from the server
	302	+ resp = self.__recvall(8)
	303	+ if resp[0] != "\x00":
	304	+ # Bad data
	305	+ self.close()
	306	+ raise GeneralProxyError((1,_generalerrors[1]))
	307	+ if resp[1] != "\x5A":
	308	+ # Server returned an error
	309	+ self.close()
	310	+ if ord(resp[1]) in (91,92,93):
	311	+ self.close()
	312	+ raise Socks4Error((ord(resp[1]),_socks4errors[ord(resp[1])-90]))
	313	+ else:
	314	+ raise Socks4Error((94,_socks4errors[4]))
	315	+ # Get the bound address/port
	316	+ self.__proxysockname = (socket.inet_ntoa(resp[4:]),struct.unpack(">H",resp[2:4])[0])
	317	+ if rmtrslv != None:
	318	+ self.__proxypeername = (socket.inet_ntoa(ipaddr),destport)
	319	+ else:
	320	+ self.__proxypeername = (destaddr,destport)
	321	+
	322	+ def __negotiatehttp(self,destaddr,destport):
	323	+ """__negotiatehttp(self,destaddr,destport)
	324	+ Negotiates a connection through an HTTP server.
	325	+ """
	326	+ # If we need to resolve locally, we do this now
	327	+ if self.__proxy[3] == False:
	328	+ addr = socket.gethostbyname(destaddr)
	329	+ else:
	330	+ addr = destaddr
	331	+ self.sendall("CONNECT " + addr + ":" + str(destport) + " HTTP/1.1\r\n" + "Host: " + destaddr + "\r\n\r\n")
	332	+ # We read the response until we get the string "\r\n\r\n"
	333	+ resp = self.recv(1)
	334	+ while resp.find("\r\n\r\n")==-1:
	335	+ resp = resp + self.recv(1)
	336	+ # We just need the first line to check if the connection
	337	+ # was successful
	338	+ statusline = resp.splitlines()[0].split(" ",2)
	339	+ if statusline[0] not in ("HTTP/1.0","HTTP/1.1"):
	340	+ self.close()
	341	+ raise GeneralProxyError((1,_generalerrors[1]))
	342	+ try:
	343	+ statuscode = int(statusline[1])
	344	+ except ValueError:
	345	+ self.close()
	346	+ raise GeneralProxyError((1,_generalerrors[1]))
	347	+ if statuscode != 200:
	348	+ self.close()
	349	+ raise HTTPError((statuscode,statusline[2]))
	350	+ self.__proxysockname = ("0.0.0.0",0)
	351	+ self.__proxypeername = (addr,destport)
	352	+
	353	+ def connect(self,destpair):
	354	+ """connect(self,despair)
	355	+ Connects to the specified destination through a proxy.
	356	+ destpar - A tuple of the IP/DNS address and the port number.
	357	+ (identical to socket's connect).
	358	+ To select the proxy server use setproxy().
	359	+ """
	360	+ # Do a minimal input check first
	361	+ if (type(destpair) in (list,tuple)==False) or (len(destpair)<2) or (type(destpair[0])!=str) or (type(destpair[1])!=int):
	362	+ raise GeneralProxyError((5,_generalerrors[5]))
	363	+ if self.__proxy[0] == PROXY_TYPE_SOCKS5:
	364	+ if self.__proxy[2] != None:
	365	+ portnum = self.__proxy[2]
	366	+ else:
	367	+ portnum = 1080
	368	+ _orgsocket.connect(self,(self.__proxy[1],portnum))
	369	+ self.__negotiatesocks5(destpair[0],destpair[1])
	370	+ elif self.__proxy[0] == PROXY_TYPE_SOCKS4:
	371	+ if self.__proxy[2] != None:
	372	+ portnum = self.__proxy[2]
	373	+ else:
	374	+ portnum = 1080
	375	+ _orgsocket.connect(self,(self.__proxy[1],portnum))
	376	+ self.__negotiatesocks4(destpair[0],destpair[1])
	377	+ elif self.__proxy[0] == PROXY_TYPE_HTTP:
	378	+ if self.__proxy[2] != None:
	379	+ portnum = self.__proxy[2]
	380	+ else:
	381	+ portnum = 8080
	382	+ _orgsocket.connect(self,(self.__proxy[1],portnum))
	383	+ self.__negotiatehttp(destpair[0],destpair[1])
	384	+ elif self.__proxy[0] == None:
	385	+ _orgsocket.connect(self,(destpair[0],destpair[1]))
	386	+ else:
	387	+ raise GeneralProxyError((4,_generalerrors[4]))
...	...