From Steven.VanVooren at esat.kuleuven.ac.be  Tue Feb 17 06:30:27 2004
From: Steven.VanVooren at esat.kuleuven.ac.be (Steven Van Vooren)
Date: Wed Feb 18 07:02:02 2004
Subject: [DAS] Couldn't instantiate datasource ensembl1834_bridge
Message-ID: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>

Dear DAS-list,

I want to serve textual annotations (from a text mining process) on clones on the human genome, and visualise these in the Ensembl genome browser web site (by entering the url to my own DAS source). For this, I am trying to install and configure a DAS server. I understand that all I need is an "annotation server" and that I should use the ensembl DAS servlet.sanger.ac.uk:8080/das as "reference server" somehow.

However, from the documentation at http://www.ensembl.org/Docs/das_server_v1.2.pdf and the docs on http://www.biojava.org/dazzle/ I gather that I should configure kaka.sanger.ac.uk as a data source for the reference server. 

Anyhow, after adapting the example configuration XML according to the pdf documentation, I get the following error message:

[11:52:47.156] DazzleServerMain: init
[11:52:47.172] org.biojava.servlets.dazzle.DazzleServlet: Error initializing installation
[11:52:47.172] org.biojava.servlets.dazzle.datasource.DataSourceException: Couldn't instantiate datasource ensembl1834_bridge
[11:52:47.172]  at org.biojava.servlets.dazzle.datasource.BasicDazzleInstallation.initResource(BasicDazzleInstallation.java:138)
...

I'm using most recent Resin and mySQL versions. As compiling the 0.99 version from the CVS repository failed, I am using the Dazzle "dazzle-webapp-skeleton-0.95" (downloaded from http://www.biojava.org/download/dazzle/). I have pasted my configuration file below. 

a) Can anyone on this list tell me what is wrong with these configuration settings? Any help would be greatly appreciated (good Karma will follow).
b) Also, is this the right way of doing things (to try to configure data bridges), or does it suffice to define a "mapmaster" pointing to the servlet.sanger.ac.uk:8080/das reference server? If so, how should I configure this?

-- config file below --

<dazzle xmlns="http://www.biojava.org/2000/dazzle">

<!-- Human data bridge -->
<resource id="ensembl1834_bridge" jclass="ensembl.EnsemblHolder">
<string name="dbURL" value="jdbc:mysql://kaka.sanger.ac.uk/homo_sapiens_core_18_34" />   (also tried 19_34a/b)
<string name="dbUser" value="anonymous" />
<string name="dbPass" value="" />
<int name="schemaVersion" value="18" /> (tried this)
<int name="schemaVersion" value="18.34" />  (and this)
<int name="schemaVersion" value="1834" />  (and this)
</resource>

<!-- Human Core data source -->
<datasource id="ensembl1834" jclass="ensembl.EnsemblReferenceSource">
<string name="ensemblHolderID" value="ensembl1834_bridge" />
<string name="name" value="Ensembl 18_34" />
<string name="description" value="Human Ensembl 1834" />
<string name="version" value="18.34" />
<set name="elideTypes">
<string value="repeat" />
<string value="similarity" />
</set>
<string name="stylesheet" value="ensemblbase-style.xml" />
</datasource>

<!-- My annotations from mySQL -->
<datasource id="MyDataSource" jclass="ensembl.EnsemblGenericSeqFeatureSource">
<string name="ensemblHolderID" value="ensembl1834_bridge" />
<string name="name" value="Ensembl Test Source" />
<string name="description" value="My DAS feature source" />
<string name="version" value="1.00" />
<string name="mapMaster" value="http://servlet.sanger.ac.uk:8080/das/ensembl1834" />
<string name="dbURL" value="jdbc:mysql://localhost/ensembl_generic" />
<string name="dbUser" value="ensembluser" />
<string name="dbPass" value="********" />
<string name="tableName" value="my_feature" />
<string name="stylesheet" value="generic-style.xml" />
</datasource>

</dazzle>

Thank you,
Steven
From ak at ebi.ac.uk  Wed Feb 18 07:37:30 2004
From: ak at ebi.ac.uk (Andreas Kahari)
Date: Wed Feb 18 07:44:23 2004
Subject: [DAS] Couldn't instantiate datasource ensembl1834_bridge
In-Reply-To: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
References: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
Message-ID: <20040218123730.GA28491@ebi.ac.uk>

On Tue, Feb 17, 2004 at 12:30:27PM +0100, Steven Van Vooren wrote:
> Dear DAS-list,
> 
> I want to serve textual annotations (from a text mining
> process) on clones on the human genome, and visualise these
> in the Ensembl genome browser web site (by entering the url
> to my own DAS source). For this, I am trying to install and
> configure a DAS server. I understand that all I need is an
> "annotation server" and that I should use the ensembl DAS
> servlet.sanger.ac.uk:8080/das as "reference server" somehow.
>
> However, from the documentation at
> http://www.ensembl.org/Docs/das_server_v1.2.pdf and the docs
> on http://www.biojava.org/dazzle/ I gather that I should
> configure kaka.sanger.ac.uk as a data source for the reference
> server.

The kaka server has been replaced by

    ensembldb.ensembl.org

I'm no Dazzle expert, but this could be your problem.

The name change was announce in August by James Stalker on
ensembl-dev and ensembl-announce:

    Hello all,

    Those of you who use our public MySQL server will
    be interested to know that it is changing name from
    kaka.sanger.ac.uk to ensembldb.ensembl.org.  Username is
    still "anonymous", there is still no password, and the port
    remains the standard 3306.  In fact, nothing has changed
    except the name.

    Both kaka and ensembldb names will work for the near
    future, but eventually the kaka hardware will be replaced
    and the kaka name may well stop working.  We intend
    ensembldb.ensembl.org to be the long-term name for this
    resource, so it's probably a good idea to start updating any
    scripts that rely on the old name.

    Please mail ensembl-dev@ebi.ac.uk if you have any questions.

    Regards,

    James



I hope this solves it.

Regards,
Andreas



-- 
|{--}|      Andreas K?h?ri                                |}{}{|
|-}{-|      EMBL, European Bioinformatics Institute       |{}{}|
|{--}|      Wellcome Trust Genome Campus, Hinxton         |}{}{|
|-}{-|      Cambridge, CB10 1SD                           |{}{}|
|{--}|      United Kingdom                                |}{}{|
From td2 at sanger.ac.uk  Wed Feb 18 07:40:13 2004
From: td2 at sanger.ac.uk (Thomas Down)
Date: Wed Feb 18 07:46:30 2004
Subject: [DAS] Couldn't instantiate datasource ensembl1834_bridge
In-Reply-To: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
References: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
Message-ID: <98F35BD8-620F-11D8-B7C4-000A95C8B056@sanger.ac.uk>


On 17 Feb 2004, at 11:30, Steven Van Vooren wrote:

> Dear DAS-list,
>
> I want to serve textual annotations (from a text mining process) on  
> clones on the human genome, and visualise these in the Ensembl genome  
> browser web site (by entering the url to my own DAS source). For this,  
> I am trying to install and configure a DAS server. I understand that  
> all I need is an "annotation server" and that I should use the ensembl  
> DAS servlet.sanger.ac.uk:8080/das as "reference server" somehow.
>
> However, from the documentation at  
> http://www.ensembl.org/Docs/das_server_v1.2.pdf and the docs on  
> http://www.biojava.org/dazzle/ I gather that I should configure  
> kaka.sanger.ac.uk as a data source for the reference server.

Both approaches ought to work.  Running your own reference server might  
be faster (and is definitely a good idea if you have your own local  
copy of the ensembl core databases).

Incidentally, the Sanger public mysql server formerly known as kaka has  
now become ensembldb.ensembl.org, and you should use that name instead.  
  Not sure if kaka still works.

> Anyhow, after adapting the example configuration XML according to the  
> pdf documentation, I get the following error message:
>
> [11:52:47.156] DazzleServerMain: init
> [11:52:47.172] org.biojava.servlets.dazzle.DazzleServlet: Error  
> initializing installation
> [11:52:47.172]  
> org.biojava.servlets.dazzle.datasource.DataSourceException: Couldn't  
> instantiate datasource ensembl1834_bridge
> [11:52:47.172]  at  
> org.biojava.servlets.dazzle.datasource.BasicDazzleInstallation.initReso 
> urce(BasicDazzleInstallation.java:138)
> ...
>
> I'm using most recent Resin and mySQL versions. As compiling the 0.99  
> version from the CVS repository failed, I am using the Dazzle  
> "dazzle-webapp-skeleton-0.95" (downloaded from  
> http://www.biojava.org/download/dazzle/). I have pasted my  
> configuration file below.

Why did compilation fail?  Which version of BioJava were you using?   
The Dazzle CVS should now be fixed to work with current  CVS  
biojava-live, but it's always possible that I didn't check the changes  
right, or that CVS is doing something strange.

The skeleton you've downloaded is very old.  You can grab something  
more current from:

       http://www.derkholm.net/thomasd/das/

(I'm currently having trouble updating the biojava.org site -- I'll  
move it over there later on).

Hope this helps,

      Thomas.

From Alex.Zelensky at anu.edu.au  Wed Feb 18 20:43:46 2004
From: Alex.Zelensky at anu.edu.au (Alex Zelensky)
Date: Wed Feb 18 20:50:48 2004
Subject: [DAS] Couldn't instantiate datasource ensembl1834_bridge
In-Reply-To: <98F35BD8-620F-11D8-B7C4-000A95C8B056@sanger.ac.uk>
References: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
	<98F35BD8-620F-11D8-B7C4-000A95C8B056@sanger.ac.uk>
Message-ID: <0EABE290-627D-11D8-9A10-000393BC6874@anu.edu.au>

Hi,

> Why did compilation fail?  Which version of BioJava were you using?  
> The Dazzle CVS should now be fixed to work with current  CVS 
> biojava-live, but it's always possible that I didn't check the changes 
> right, or that CVS is doing something strange.

I also had problems compiling dazzle from cvs with Tomcat 4. My 
explanation was that dazzle implements Java Servlet 2.2 specification, 
while  Tomcat 4 implements Java Servlet 2.3. I was able to compile 
dazzle using servlet.jar from Tomcat 3.

Regards,
Alex

From Steven.VanVooren at esat.kuleuven.ac.be  Thu Feb 19 15:24:17 2004
From: Steven.VanVooren at esat.kuleuven.ac.be (Steven Van Vooren)
Date: Thu Feb 19 15:30:32 2004
Subject: [DAS] Couldn't instantiate datasource ensembl1834_bridge
References: <009201c3f549$715995a0$5484210a@esat.kuleuven.ac.be>
	<98F35BD8-620F-11D8-B7C4-000A95C8B056@sanger.ac.uk>
Message-ID: <018a01c3f726$596009f0$5484210a@esat.kuleuven.ac.be>

Dear DAS-list,

First of all, thanks for your comments. Things seem to be working fine now.
I'm now posting some additional questions that have come up.

> Why did compilation fail?  Which version of BioJava were you using?

Using your recent compilation at http://www.derkholm.net/thomasd/das/
(thanks), things are working just fine, so there was no need for me to
compile Dazzle anymore. However, I'm posting this for follow up. After
switching to servlets.jar from apache tomcat 3.3.1a (thanks, Alex), after
grabbing biojava-live from the cvs repository and successfully compiling it,
and after putting everything in the classpath, I still got:

-------- output -------- 
compile:
    [javac] Compiling 20 source files to D:\biodas\xdas\ant-build\classes
    [javac]
D:\biodas\xdas\ant-build\src\org\biojava\servlets\dazzle\datasource\RagbagDa
taSource.java:31: package org.biojava.bio.seq.ragbag does not exist
    [javac] import org.biojava.bio.seq.ragbag.*;
    [javac] ^
[ ... other errors resulting from this ... ]
    [javac]
D:\biodas\xdas\ant-build\src\org\biojava\servlets\dazzle\virtual\VirtualData
Source.java:53: org.biojava.servlets.dazzle.virtual.VirtualDataSource is not
abstract and does not override abstract method
getShatterFeature(org.biojava.bio.seq.Feature) in
org.biojava.servlets.dazzle.datasource.DazzleDataSource
    [javac] class VirtualDataSource implements DazzleDataSource {
    [javac] ^
[ ... other errors resulting from this ... ]
-------- output -------- 

The ragbag package is indeed not present in the biojava-live jar.

> The Dazzle CVS should now be fixed to work with current  CVS
> biojava-live, but it's always possible that I didn't check the changes
> right, or that CVS is doing something strange.

Maybe I'm not using the correct sources. I checked out xdas from the biojava
CVS, I hope this is the correct source for the latest dazzle
(http://cvs.biojava.org/cgi-bin/viewcvs/viewcvs.cgi/xdas/?cvsroot=biojava,
checked out  using cvs -d
:pserver:cvs@cvs.open-bio.org:/home/repository/biojava checkout xdas).

> The skeleton you've downloaded is very old.  You can grab something
> more current from:
>
>        http://www.derkholm.net/thomasd/das/

Thanks for putting this online, it works fine for me. But some new questions
have now come up. Maybe someone on the list can help me out.

[Question 1 - feature storage]

I have set up a MySQL database as described in
http://www.ensembl.org/Docs/das_server_v1.2.pdf (page 4) and configured
dazzle to read annotations from it, which works. In the example described in
that pdf, the contig_id field of the annotations table contains contigs
names (AC012170.6.1.198550). I have tried adding contigs there myself, but
they do not show up when I use the ensembl genome browser to visualise the
region surrounding that contig. Instead, ensembl just reports "no * My...
features in this region".

However, when I put a chromosome number in the contig_id field of that
table, things work fine, and I get the ensembl genome browser to show my
annotations. I assume this is due to the fact that dazzle does not "know"
contig names as entry points (looking at
http://servlet.sanger.ac.uk:8080/das/ensembl1834/entry_points I indeed only
get chromosome numbers). Am I right in this assumption? Is there a way to
enable dazzle to use contig and clone names, instead of putting my
annotations in the database using an offset relative to the "first basepair"
of each chromosome? In the example table in
http://www.ensembl.org/Docs/das_server_v1.2.pdf , they seem to do exactly
that. As the annotations I want to serve for now, coincide perfectly to
clones/contigs, this would save me the conversion step to chromosome
positions.

[Question 2 - links]

>From the LDAS (perl based das server) documentation, it seems to be possible
to serve http-links with annotations. There, a config file tells LDAS how to
construct links from the annotation data. I assume this makes annotations
clickable in the ensembl genome browser. Can I also do this with Dazzle, and
if so, does it require me to add code to the datasource class and compile
them myself, or is there an easy way to serve links? Allowing visitors to
click-through from ensembl to detailed information on the annotations would
be useful in my case.

[Question 3 - speed]

> Both approaches ought to work.  Running your own reference server might
> be faster (and is definitely a good idea if you have your own local
> copy of the ensembl core databases).

In order to run my own reference server, would it suffice just to copy the
homo_sapiens_core_19_34a tables to a local database and point to it in the
config file ( <string name="dbURL"
value="jdbc:mysql://mylocaldbserver/homo_sapiens_core_19_34a" /> ) or are
there other issues to consider?

[Question 4 - caching]

After inserting new annotations to the feature table in MySQL (additionally
to those that were already in the database) the ensembl site only shows
those annotations that were already there. Refreshing the browser window
doesn't solve this. It seems that somewhere, annotations are cached. Is this
done at the ensembl site, or does dazzle cache the annotations itself? How
should can you purge this and force a database-read to include newly added
annotations?

[Question 5 - XML]

1. What is the effect of <boolean name="useHeavyClones" value="true" /> ?

2. I understand that I can use <set name="elideTypes"></set> and <set
name="acceptTypes"></set> to avoid specific feature types to be served. If I
just want dazzle to act as an annotation server with my features, can I use
these settings somehow to speed things up?

[As a reference, this is the XML file I use.]

<dazzle xmlns="http://www.biojava.org/2000/dazzle">

  <resource id="ensembl1934a_bridge" jclass="ensembl.EnsemblHolder">
    <string name="dbURL"
value="jdbc:mysql://ensembldb.ensembl.org/homo_sapiens_core_19_34a" />
    <string name="dbUser" value="anonymous" />
    <string name="dbPass" value="" />
    <boolean name="useHeavyClones" value="true" />

    <int name="schemaVersion" value="19" />
  </resource>

  <datasource id="ensembl1934a" jclass="ensembl.EnsemblReferenceSource">
    <string name="ensemblHolderID" value="ensembl1934a_bridge" />
    <string name="name" value="Ensembl 19.34a" />
    <string name="description" value="The human genome from Ensembl" />
    <string name="version" value="19.34a" />
    <set name="elideTypes">
      <string value="repeat" />
      <string value="similarity" />
      <string value="TSS" />
    </set>
    <string name="stylesheet" value="ensemblbase-style.xml" />
    <string name="ensemblWebURL"
value="http://www.ensembl.org/Homo_sapiens/" />
  </datasource>

<!-- My annotations from mySQL -->
<datasource id="MyDataSource"
jclass="ensembl.EnsemblGenericSeqFeatureSource">
<string name="ensemblHolderID" value="ensembl1934a_bridge" />
<string name="name" value="Ensembl Test Source" />
<string name="description" value="My DAS feature source" />
<string name="version" value="1.00" />
<string name="mapMaster"
value="http://servlet.sanger.ac.uk:8080/das/ensembl1934a" />
<string name="dbURL" value="jdbc:mysql://mysqlhost/ensembl_generic" />
<string name="dbUser" value="****" />
<string name="dbPass" value="****" />
<string name="tableName" value="my_feature" />
<string name="stylesheet" value="generic-style.xml" />
</datasource>

</dazzle>

Thank you,
Steven