From mitch_skinner at berkeley.edu Fri Oct 1 15:57:04 2010 From: mitch_skinner at berkeley.edu (Mitch Skinner) Date: Fri, 01 Oct 2010 12:57:04 -0700 Subject: [DAS] 1.6 draft 7 In-Reply-To: <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> References: <133B31E7-063A-4D82-9938-4D42B0A93DEF@ebi.ac.uk> <4CA3053F.6020400@berkeley.edu> <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> Message-ID: <4CA63D10.5030700@berkeley.edu> 09/29/2010 08:57 AM, Andy Jenkinson wrote: > In fact I did a little test: I randomly generated 100,000 features in a JSON format with 5 small string fields (indexed file size ~5 mb). It turns out that when uncompressed a keyed file is indeed much bigger (87.5%). After compression this went down to 25.8% though. When 10% of fields were empty (i.e. for the indexed style empty strings, and for the hashed style omitted pairs), this went down again to 16.8%. This last bit surprised me to be honest, I expected a much more modest effect. For 10% of fields in a dataset to be empty seems a reasonable expectation to me too, so if your data model has a variable "occupancy" of fields across rows it's worth considering I would suggest. In particular, using 'null' instead of empty strings would have an even bigger effect (I don't know what JBrowse does in such circumstances). Hi, thanks for the thoughtful and detailed analysis; it's really nice to get numbers on this stuff. 16.8% is certainly less than I would have hoped for, but I think there may be some ways to win back a bit more space. In JBrowse, the "schema" can vary by track; my assumption was that the set of populated attributes in an individual track would be pretty uniform. Some tracks might not use the "phase" field, for example, but if a given track used phase information, then I figured that almost all of the features in that track would populate that field. For DAS, I was assuming that the "schema" could similarly vary on a per-type or per-query basis. In other words, I think if it being something that could be determined at server set-up time, or HTTP-request time rather than being fixed in the DAS standard (the standard could specify a controlled vocabulary of required/optional fields, but it doesn't have to specify array indexes). I'm not sure how easy that would be to implement; I imagine it would be relatively straightforward for someone using something like the UCSC genome browser database schema to determine the fields used in a given query result based on the columns in the table being queried. But for other types of DB schemas I imagine it could be harder to know what fields are going to be used, unless you write code to go through the query result and check. And I'm sure you're right that using null rather than an empty string for unpopulated fields would use a bit more space. One possibility would be to put the fields that are more likely to be empty at the end, and just make the array shorter if a given record doesn't have values for the trailing fields. Also, javascript allows for array entries to be omitted entirely, like: [10000, 15000, , "foo"] JSON theoretically doesn't allow this; omitted entries become "undefined" in javascript, and the "official" JSON spec disallows "undefined" in an effort to facilitate interoperation with languages that don't have separate notions of "null" and "undefined" the way javascript does. But I'm not sure how often people obey that restriction in practice; I personally think it was a silly choice. If JSON doesn't want to include separate notions of "null" and "undefined", then it can specify that they both map to the same thing in other languages; e.g., in java they could both map to java's null. I'd feel comfortable writing code that ignored that restriction and allowed for omitted array entries. If you did that, then the cost of an empty field is just one byte for the comma, or nothing if the field is at the end of an array and you just send a shorter array. Also, depending on the use case, I wonder if the difference in (de)compression time between indexed and keyed JSON would matter. If you have your generated data handy still, I'd be curious to know what the difference is. > Regarding memory, if you think about it there is nothing to stop you using arrays in code regardless of the file format, should memory be an issue. True, although when I started I was paranoid about reducing the amount of work done in javascript in the browser, so I wanted to pre-digest things if I could rather than having the client translate. Over the last few years, web browsers have gotten so much faster, but I think it's still important to support IE; a significant number of people will be using slow versions of IE (6-8) for a long time to come. I recognize that DAS has a much broader variety of clients than just web browsers, though. Regards, Mitch From thomas.a.down at gmail.com Sat Oct 2 11:11:38 2010 From: thomas.a.down at gmail.com (Thomas Down) Date: Sat, 2 Oct 2010 16:11:38 +0100 Subject: [DAS] 1.6 draft 7 In-Reply-To: <1D0FE721-66B3-4402-8F4F-A2F30E5F2EDD@ebi.ac.uk> References: <1D0FE721-66B3-4402-8F4F-A2F30E5F2EDD@ebi.ac.uk> Message-ID: On Wed, Sep 22, 2010 at 4:44 PM, Andy Jenkinson wrote: > Hi all, > > I have updated the 1.6 specification to draft 7 in light of recent > discussions on the list: Hi Andy, thanks for all your work on this. I'm pleased to announce that Dalliance 0.5.2 should now be a more-or-less fully functional DAS 1.6 client, and I hope this will be useful to anyone who's testing out DAS 1.6 server or datasource implementations -- just paste your datasource URL into the add custom track dialog and it should work (fingers crossed!). The new release is available now at: http://www.biodalliance.org/ Thomas. From andy.jenkinson at ebi.ac.uk Mon Oct 4 04:54:38 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 4 Oct 2010 09:54:38 +0100 Subject: [DAS] 1.6 draft 7 In-Reply-To: <4CA63D10.5030700@berkeley.edu> References: <133B31E7-063A-4D82-9938-4D42B0A93DEF@ebi.ac.uk> <4CA3053F.6020400@berkeley.edu> <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> <4CA63D10.5030700@berkeley.edu> Message-ID: On 1 Oct 2010, at 20:57, Mitch Skinner wrote: > In JBrowse, the "schema" can vary by track; my assumption was that the set of populated attributes in an individual track would be pretty uniform. Some tracks might not use the "phase" field, for example, but if a given track used phase information, then I figured that almost all of the features in that track would populate that field. Indeed, and so long as your server/data file can determine in advance that phase is not used at all (as you say), it can omit it entirely. I'd say most of the time that's going to be perfectly possible. And to be honest I'm not wholly convinced that DAS has significantly less uniformity in the fields used within a data set, but it's worth us bearing in mind. > Also, javascript allows for array entries to be omitted entirely, like: > > [10000, 15000, , "foo"] > > JSON theoretically doesn't allow this; omitted entries become "undefined" in javascript, and the "official" JSON spec disallows "undefined" Interesting, I must admit my use of JSON is fairly limited so I wasn't aware of this possibility. It doesn't sound "nice", but when you're on the edge trying to squeeze out all the speed you can, it doesn't seem an issue. > > Also, depending on the use case, I wonder if the difference in (de)compression time between indexed and keyed JSON would matter. If you have your generated data handy still, I'd be curious to know what the difference is. I don't have the original files, but I do have the script to generate some more similar ones (attached). You could play around with whitespace too if you want to do some optimisation. -------------- next part -------------- A non-text attachment was scrubbed... Name: test-filesize.pl Type: text/x-perl-script Size: 1117 bytes Desc: not available URL: -------------- next part -------------- I'm sure there is a lot of existing study behind this very question. My expectation would be that the overhead of compression/decompression is going to be worth it for anything larger than 100 kb or so (probably even less), such is the huge difference it makes to the size of these files and bandwidth is usually the rate-limiting step. Up to now I have always considered that the chances are, if you're worried about speed and the size of your files, you probably need compression. Cheers, Andy From andy.jenkinson at ebi.ac.uk Wed Oct 6 12:27:38 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Wed, 6 Oct 2010 17:27:38 +0100 Subject: [DAS] 1.6 specification release Message-ID: Hi all, Just a quick note to say that since the last draft of 1.6, only clarifications to the text have been required and therefore the specification is considered fit for release. This is due to happen on Tuesday 19th October (I will be away from now until the 18th). On this date, although future revisions to the specification text are possible, the meaning and functionality will be frozen. The BioDAS website will be updated and an information page will be in place to help developers of clients and servers deal with any version compatibility issues arising from the switchover. In particular a list of clients and projected dates for 1.6 support. More details to follow in due course. Cheers, Andy From jw12 at sanger.ac.uk Thu Oct 7 07:08:09 2010 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 7 Oct 2010 12:08:09 +0100 Subject: [DAS] Removal of http://das.ensembl.org/das Message-ID: <0EC10E42-E483-4594-B55C-6A63FC6D8FCD@sanger.ac.uk> Hi All sources in the DAS registry which referred to http://das.ensembl.org/das/ now refer to http://das.sanger.ac.uk/das/ From now on please DO NOT refer to Sanger DAS sources with the old das.ensembl.org url as these DAS sources no longer show in ensembl. In the future only DAS sources served from the main ensembl/ensembl archive databases will be available from das.ensembl.org. Any problems please let me know. Sorry for any inconvenience caused but this had to be done due to technical reasons and as part of the Sanger DAS sources cleanup. Thanks Jonathan. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From andy.jenkinson at ebi.ac.uk Tue Oct 19 14:47:18 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Tue, 19 Oct 2010 19:47:18 +0100 Subject: [DAS] 1.6 specification release In-Reply-To: References: Message-ID: <5CE9BBBF-8638-49B0-B018-5DF627DB9676@ebi.ac.uk> The 1.6 specification is now online, and the BioDas homepage has been updated: http://www.biodas.org/ http://www.biodas.org/wiki/DAS1.6 http://www.biodas.org/wiki/DAS_1.6_Migration If you are a client or server developer, it'd be useful if you could fill in on the migration page any information you can regarding timescales for 1.6 support in your application. Please let me know if you spot any links/text on the website erroneously referring to the old spec (or just edit it yourself). Cheers, Andy On 6 Oct 2010, at 17:27, Andy Jenkinson wrote: > Hi all, > > Just a quick note to say that since the last draft of 1.6, only clarifications to the text have been required and therefore the specification is considered fit for release. This is due to happen on Tuesday 19th October (I will be away from now until the 18th). On this date, although future revisions to the specification text are possible, the meaning and functionality will be frozen. The BioDAS website will be updated and an information page will be in place to help developers of clients and servers deal with any version compatibility issues arising from the switchover. In particular a list of clients and projected dates for 1.6 support. > > More details to follow in due course. > > Cheers, > Andy > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Fri Oct 29 04:55:00 2010 From: dan.bolser at gmail.com (Dan Bolser) Date: Fri, 29 Oct 2010 10:55:00 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: <4CC71E8F.8020802@stanford.edu> References: <4CC71E8F.8020802@stanford.edu> Message-ID: Hi Bob, This is a nice idea. In my spare time I've been working on running a DAS server from a MediaWiki data source. (MediaWiki is the wiki that runs Wikipedia). In theory, you could have a comments track that was pulled in from the 'DASWiki', and clicking the details link would take you to the wiki page to edit the comment. Going back to the GBrowse, you would then see the updated data from the wiki page. It wouldn't be too hard to have a similar workflow for adding new comments. I saw a talk from some people at KeyGene who are doing something similar (loading DAS into a wiki and linking it to GBrowse to generate query reports for specific regions). You can see the talk here: SMWCon Fall 2010 / Sunday / Rudi van Bavel * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g ** http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv If anyone wants to take what I have done so far, my (sadly neglected) prototype is here: http://das.referata.com/wiki/Main_Page Kick me a few times and I'll get the underlying code into shape. A year or two back I did have a working prototype based on a MediaWiki ProServer adaptor. If you are interested in community annotation should think about attending this: Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: http://www.nettab.org/2010/ Cheers, Dan. On 26 October 2010 20:31, Bob Muller wrote: > We are working with a small research project that doesn't have funds for a full-scale database for features, so they want to know: > > How difficult would it be to add a comment section editable by users to gbrowse? ?When you click on a feature in gbrowse it shows a detail page for that feature, is there any way we could enable edits that could be written back to the underlying mysql database that gbrowse runs on? > > Their idea is to use the SeqFeature::Store database to hold "comments" entered through GBrowse as a way to annotate their genome. I'd be interested in whether this is possible in GBrowse 1.7 or in 2.x. > > ? --Bob Muller, TAIR (www.arabidopsis.org) > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in ?U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > Gmod-gbrowse mailing list > Gmod-gbrowse at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse > From gsalazar at cs.uct.ac.za Fri Oct 29 05:58:59 2010 From: gsalazar at cs.uct.ac.za (Gustavo Adolfo Salazar Orejuela) Date: Fri, 29 Oct 2010 11:58:59 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: Hello There, I have been working in an extension to DAS1.6 called writeback( http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow to create new versions of an existing feature. So, putting new notes into a feature is just a creation of a new version of the feature. The server implementation of the writeback extends MyDas (http://code.google.com/p/mydas/) and you can get a mydas+writebackk datasource by SVN ( https://mydas.googlecode.com/svn/writeback-datasource). We are close to release the plugin for Dasty3 to use the writeback server, however this first client is a protein client, and so far there is not a development for a genome client with DAS writeback capability, but I am quite interested in this. I hope to be able to announce the release of the writeback client in Dasty3 in a few days, so you will be able to see it working and see if this approach goes in the same direction of what you are looking for. Regards, Gustavo. 2010/10/29 Dan Bolser > Hi Bob, > > This is a nice idea. In my spare time I've been working on running a > DAS server from a MediaWiki data source. (MediaWiki is the wiki that > runs Wikipedia). > > In theory, you could have a comments track that was pulled in from the > 'DASWiki', and clicking the details link would take you to the wiki > page to edit the comment. Going back to the GBrowse, you would then > see the updated data from the wiki page. It wouldn't be too hard to > have a similar workflow for adding new comments. > > I saw a talk from some people at KeyGene who are doing something > similar (loading DAS into a wiki and linking it to GBrowse to generate > query reports for specific regions). You can see the talk here: > > SMWCon Fall 2010 / Sunday / Rudi van Bavel > * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program > ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g > ** > http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv > > > If anyone wants to take what I have done so far, my (sadly neglected) > prototype is here: > > http://das.referata.com/wiki/Main_Page > > > Kick me a few times and I'll get the underlying code into shape. A > year or two back I did have a working prototype based on a MediaWiki > ProServer adaptor. > > > If you are interested in community annotation should think about attending > this: > > Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: > http://www.nettab.org/2010/ > > > Cheers, > Dan. > > On 26 October 2010 20:31, Bob Muller wrote: > > We are working with a small research project that doesn't have funds for > a full-scale database for features, so they want to know: > > > > How difficult would it be to add a comment section editable by users to > gbrowse? When you click on a feature in gbrowse it shows a detail page for > that feature, is there any way we could enable edits that could be written > back to the underlying mysql database that gbrowse runs on? > > > > Their idea is to use the SeqFeature::Store database to hold "comments" > entered through GBrowse as a way to annotate their genome. I'd be interested > in whether this is possible in GBrowse 1.7 or in 2.x. > > > > --Bob Muller, TAIR (www.arabidopsis.org) > > > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > > Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > Gmod-gbrowse mailing list > > Gmod-gbrowse at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse > > > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das > -- Gustavo Adolfo Salazar Orejuela From andy.jenkinson at ebi.ac.uk Fri Oct 29 10:14:21 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Fri, 29 Oct 2010 15:14:21 +0100 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: <9B1B41B5-9200-4CD2-9667-45F342E57EF3@ebi.ac.uk> We also run a simple ProServer SourceAdaptor to pull in content from SNPedia, using the Perlwikipedia CPAN module. The solution of sending a user to a wiki sounds somewhat clunky to me, but would be relatively easy to set up I imagine. As Gustavo says, a full implementation of writeback in a production system from client through to backend is very close. Dasty3 (client) will be able to edit features directly in the interface, with the edits being stored in a DAS source separate from the original feature. Something similar could certainly be implemented for GBrowse, but would require development work. But as compensation, no development of the backend would be required. Cheers, Andy On 29 Oct 2010, at 09:55, Dan Bolser wrote: > Hi Bob, > > This is a nice idea. In my spare time I've been working on running a > DAS server from a MediaWiki data source. (MediaWiki is the wiki that > runs Wikipedia). > > In theory, you could have a comments track that was pulled in from the > 'DASWiki', and clicking the details link would take you to the wiki > page to edit the comment. Going back to the GBrowse, you would then > see the updated data from the wiki page. It wouldn't be too hard to > have a similar workflow for adding new comments. > > I saw a talk from some people at KeyGene who are doing something > similar (loading DAS into a wiki and linking it to GBrowse to generate > query reports for specific regions). You can see the talk here: > > SMWCon Fall 2010 / Sunday / Rudi van Bavel > * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program > ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g > ** http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv > > > If anyone wants to take what I have done so far, my (sadly neglected) > prototype is here: > > http://das.referata.com/wiki/Main_Page > > > Kick me a few times and I'll get the underlying code into shape. A > year or two back I did have a working prototype based on a MediaWiki > ProServer adaptor. > > > If you are interested in community annotation should think about attending this: > > Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: > http://www.nettab.org/2010/ > > > Cheers, > Dan. > > On 26 October 2010 20:31, Bob Muller wrote: >> We are working with a small research project that doesn't have funds for a full-scale database for features, so they want to know: >> >> How difficult would it be to add a comment section editable by users to gbrowse? When you click on a feature in gbrowse it shows a detail page for that feature, is there any way we could enable edits that could be written back to the underlying mysql database that gbrowse runs on? >> >> Their idea is to use the SeqFeature::Store database to hold "comments" entered through GBrowse as a way to annotate their genome. I'd be interested in whether this is possible in GBrowse 1.7 or in 2.x. >> >> --Bob Muller, TAIR (www.arabidopsis.org) >> >> ------------------------------------------------------------------------------ >> Nokia and AT&T present the 2010 Calling All Innovators-North America contest >> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> http://p.sf.net/sfu/nokia-dev2dev >> _______________________________________________ >> Gmod-gbrowse mailing list >> Gmod-gbrowse at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >> > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Fri Oct 29 11:28:04 2010 From: dan.bolser at gmail.com (Dan Bolser) Date: Fri, 29 Oct 2010 17:28:04 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: My thinking was that the DASWiki approach could also support the DAS writeback extension. On 29 October 2010 11:58, Gustavo Adolfo Salazar Orejuela wrote: > Hello There, > > I have been working in an extension to DAS1.6 called > writeback(http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow > to create new versions of an existing feature. So, putting new notes into a > feature is just a creation of a new version of the feature. The server > implementation of the writeback extends MyDas > (http://code.google.com/p/mydas/) and you can get a mydas+writebackk > datasource by SVN (https://mydas.googlecode.com/svn/writeback-datasource). > We are close to release the plugin for Dasty3 to use the writeback server, > however this first client is a protein client, and so far there is not a > development for a genome client with DAS writeback capability, but I am > quite interested in this. > I hope to be able to announce the release of the writeback client in Dasty3 > in a few days, so you will be able to see it working and see if this > approach goes in the same direction of what you are looking for. > > Regards, > > Gustavo. > > 2010/10/29 Dan Bolser >> >> Hi Bob, >> >> This is a nice idea. In my spare time I've been working on running a >> DAS server from a MediaWiki data source. (MediaWiki is the wiki that >> runs Wikipedia). >> >> In theory, you could have a comments track that was pulled in from the >> 'DASWiki', and clicking the details link would take you to the wiki >> page to edit the comment. Going back to the GBrowse, you would then >> see the updated data from the wiki page. It wouldn't be too hard to >> have a similar workflow for adding new comments. >> >> I saw a talk from some people at KeyGene who are doing something >> similar (loading DAS into a wiki and linking it to GBrowse to generate >> query reports for specific regions). You can see the talk here: >> >> SMWCon Fall 2010 / Sunday / Rudi van Bavel >> * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program >> ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g >> ** >> http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv >> >> >> If anyone wants to take what I have done so far, my (sadly neglected) >> prototype is here: >> >> http://das.referata.com/wiki/Main_Page >> >> >> Kick me a few times and I'll get the underlying code into shape. A >> year or two back I did have a working prototype based on a MediaWiki >> ProServer adaptor. >> >> >> If you are interested in community annotation should think about attending >> this: >> >> Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: >> http://www.nettab.org/2010/ >> >> >> Cheers, >> Dan. >> >> On 26 October 2010 20:31, Bob Muller wrote: >> > We are working with a small research project that doesn't have funds for >> > a full-scale database for features, so they want to know: >> > >> > How difficult would it be to add a comment section editable by users to >> > gbrowse? ?When you click on a feature in gbrowse it shows a detail page for >> > that feature, is there any way we could enable edits that could be written >> > back to the underlying mysql database that gbrowse runs on? >> > >> > Their idea is to use the SeqFeature::Store database to hold "comments" >> > entered through GBrowse as a way to annotate their genome. I'd be interested >> > in whether this is possible in GBrowse 1.7 or in 2.x. >> > >> > ? --Bob Muller, TAIR (www.arabidopsis.org) >> > >> > >> > ------------------------------------------------------------------------------ >> > Nokia and AT&T present the 2010 Calling All Innovators-North America >> > contest >> > Create new apps & games for the Nokia N8 for consumers in ?U.S. and >> > Canada >> > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >> > marketing >> > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> > http://p.sf.net/sfu/nokia-dev2dev >> > _______________________________________________ >> > Gmod-gbrowse mailing list >> > Gmod-gbrowse at lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >> > >> >> _______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das > > > > -- > Gustavo Adolfo Salazar Orejuela > From andy.jenkinson at ebi.ac.uk Sat Oct 30 07:41:41 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Sat, 30 Oct 2010 12:41:41 +0100 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: <6631318A-5425-417F-8220-4627B0116C36@ebi.ac.uk> Hi Dan, Sounds interesting. How do you see that working? Cheers, Andy On 29 Oct 2010, at 16:28, Dan Bolser wrote: > My thinking was that the DASWiki approach could also support the DAS > writeback extension. > > On 29 October 2010 11:58, Gustavo Adolfo Salazar Orejuela > wrote: >> Hello There, >> >> I have been working in an extension to DAS1.6 called >> writeback(http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow >> to create new versions of an existing feature. So, putting new notes into a >> feature is just a creation of a new version of the feature. The server >> implementation of the writeback extends MyDas >> (http://code.google.com/p/mydas/) and you can get a mydas+writebackk >> datasource by SVN (https://mydas.googlecode.com/svn/writeback-datasource). >> We are close to release the plugin for Dasty3 to use the writeback server, >> however this first client is a protein client, and so far there is not a >> development for a genome client with DAS writeback capability, but I am >> quite interested in this. >> I hope to be able to announce the release of the writeback client in Dasty3 >> in a few days, so you will be able to see it working and see if this >> approach goes in the same direction of what you are looking for. >> >> Regards, >> >> Gustavo. >> >> 2010/10/29 Dan Bolser >>> >>> Hi Bob, >>> >>> This is a nice idea. In my spare time I've been working on running a >>> DAS server from a MediaWiki data source. (MediaWiki is the wiki that >>> runs Wikipedia). >>> >>> In theory, you could have a comments track that was pulled in from the >>> 'DASWiki', and clicking the details link would take you to the wiki >>> page to edit the comment. Going back to the GBrowse, you would then >>> see the updated data from the wiki page. It wouldn't be too hard to >>> have a similar workflow for adding new comments. >>> >>> I saw a talk from some people at KeyGene who are doing something >>> similar (loading DAS into a wiki and linking it to GBrowse to generate >>> query reports for specific regions). You can see the talk here: >>> >>> SMWCon Fall 2010 / Sunday / Rudi van Bavel >>> * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program >>> ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g >>> ** >>> http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv >>> >>> >>> If anyone wants to take what I have done so far, my (sadly neglected) >>> prototype is here: >>> >>> http://das.referata.com/wiki/Main_Page >>> >>> >>> Kick me a few times and I'll get the underlying code into shape. A >>> year or two back I did have a working prototype based on a MediaWiki >>> ProServer adaptor. >>> >>> >>> If you are interested in community annotation should think about attending >>> this: >>> >>> Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: >>> http://www.nettab.org/2010/ >>> >>> >>> Cheers, >>> Dan. >>> >>> On 26 October 2010 20:31, Bob Muller wrote: >>>> We are working with a small research project that doesn't have funds for >>>> a full-scale database for features, so they want to know: >>>> >>>> How difficult would it be to add a comment section editable by users to >>>> gbrowse? When you click on a feature in gbrowse it shows a detail page for >>>> that feature, is there any way we could enable edits that could be written >>>> back to the underlying mysql database that gbrowse runs on? >>>> >>>> Their idea is to use the SeqFeature::Store database to hold "comments" >>>> entered through GBrowse as a way to annotate their genome. I'd be interested >>>> in whether this is possible in GBrowse 1.7 or in 2.x. >>>> >>>> --Bob Muller, TAIR (www.arabidopsis.org) >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Nokia and AT&T present the 2010 Calling All Innovators-North America >>>> contest >>>> Create new apps & games for the Nokia N8 for consumers in U.S. and >>>> Canada >>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >>>> marketing >>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >>>> http://p.sf.net/sfu/nokia-dev2dev >>>> _______________________________________________ >>>> Gmod-gbrowse mailing list >>>> Gmod-gbrowse at lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >>>> >>> >>> _______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das >> >> >> >> -- >> Gustavo Adolfo Salazar Orejuela >> > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From mitch_skinner at berkeley.edu Fri Oct 1 19:57:04 2010 From: mitch_skinner at berkeley.edu (Mitch Skinner) Date: Fri, 01 Oct 2010 12:57:04 -0700 Subject: [DAS] 1.6 draft 7 In-Reply-To: <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> References: <133B31E7-063A-4D82-9938-4D42B0A93DEF@ebi.ac.uk> <4CA3053F.6020400@berkeley.edu> <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> Message-ID: <4CA63D10.5030700@berkeley.edu> 09/29/2010 08:57 AM, Andy Jenkinson wrote: > In fact I did a little test: I randomly generated 100,000 features in a JSON format with 5 small string fields (indexed file size ~5 mb). It turns out that when uncompressed a keyed file is indeed much bigger (87.5%). After compression this went down to 25.8% though. When 10% of fields were empty (i.e. for the indexed style empty strings, and for the hashed style omitted pairs), this went down again to 16.8%. This last bit surprised me to be honest, I expected a much more modest effect. For 10% of fields in a dataset to be empty seems a reasonable expectation to me too, so if your data model has a variable "occupancy" of fields across rows it's worth considering I would suggest. In particular, using 'null' instead of empty strings would have an even bigger effect (I don't know what JBrowse does in such circumstances). Hi, thanks for the thoughtful and detailed analysis; it's really nice to get numbers on this stuff. 16.8% is certainly less than I would have hoped for, but I think there may be some ways to win back a bit more space. In JBrowse, the "schema" can vary by track; my assumption was that the set of populated attributes in an individual track would be pretty uniform. Some tracks might not use the "phase" field, for example, but if a given track used phase information, then I figured that almost all of the features in that track would populate that field. For DAS, I was assuming that the "schema" could similarly vary on a per-type or per-query basis. In other words, I think if it being something that could be determined at server set-up time, or HTTP-request time rather than being fixed in the DAS standard (the standard could specify a controlled vocabulary of required/optional fields, but it doesn't have to specify array indexes). I'm not sure how easy that would be to implement; I imagine it would be relatively straightforward for someone using something like the UCSC genome browser database schema to determine the fields used in a given query result based on the columns in the table being queried. But for other types of DB schemas I imagine it could be harder to know what fields are going to be used, unless you write code to go through the query result and check. And I'm sure you're right that using null rather than an empty string for unpopulated fields would use a bit more space. One possibility would be to put the fields that are more likely to be empty at the end, and just make the array shorter if a given record doesn't have values for the trailing fields. Also, javascript allows for array entries to be omitted entirely, like: [10000, 15000, , "foo"] JSON theoretically doesn't allow this; omitted entries become "undefined" in javascript, and the "official" JSON spec disallows "undefined" in an effort to facilitate interoperation with languages that don't have separate notions of "null" and "undefined" the way javascript does. But I'm not sure how often people obey that restriction in practice; I personally think it was a silly choice. If JSON doesn't want to include separate notions of "null" and "undefined", then it can specify that they both map to the same thing in other languages; e.g., in java they could both map to java's null. I'd feel comfortable writing code that ignored that restriction and allowed for omitted array entries. If you did that, then the cost of an empty field is just one byte for the comma, or nothing if the field is at the end of an array and you just send a shorter array. Also, depending on the use case, I wonder if the difference in (de)compression time between indexed and keyed JSON would matter. If you have your generated data handy still, I'd be curious to know what the difference is. > Regarding memory, if you think about it there is nothing to stop you using arrays in code regardless of the file format, should memory be an issue. True, although when I started I was paranoid about reducing the amount of work done in javascript in the browser, so I wanted to pre-digest things if I could rather than having the client translate. Over the last few years, web browsers have gotten so much faster, but I think it's still important to support IE; a significant number of people will be using slow versions of IE (6-8) for a long time to come. I recognize that DAS has a much broader variety of clients than just web browsers, though. Regards, Mitch From thomas.a.down at gmail.com Sat Oct 2 15:11:38 2010 From: thomas.a.down at gmail.com (Thomas Down) Date: Sat, 2 Oct 2010 16:11:38 +0100 Subject: [DAS] 1.6 draft 7 In-Reply-To: <1D0FE721-66B3-4402-8F4F-A2F30E5F2EDD@ebi.ac.uk> References: <1D0FE721-66B3-4402-8F4F-A2F30E5F2EDD@ebi.ac.uk> Message-ID: On Wed, Sep 22, 2010 at 4:44 PM, Andy Jenkinson wrote: > Hi all, > > I have updated the 1.6 specification to draft 7 in light of recent > discussions on the list: Hi Andy, thanks for all your work on this. I'm pleased to announce that Dalliance 0.5.2 should now be a more-or-less fully functional DAS 1.6 client, and I hope this will be useful to anyone who's testing out DAS 1.6 server or datasource implementations -- just paste your datasource URL into the add custom track dialog and it should work (fingers crossed!). The new release is available now at: http://www.biodalliance.org/ Thomas. From andy.jenkinson at ebi.ac.uk Mon Oct 4 08:54:38 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 4 Oct 2010 09:54:38 +0100 Subject: [DAS] 1.6 draft 7 In-Reply-To: <4CA63D10.5030700@berkeley.edu> References: <133B31E7-063A-4D82-9938-4D42B0A93DEF@ebi.ac.uk> <4CA3053F.6020400@berkeley.edu> <8E2B9E01-9714-4FC3-8FE5-611F37F93607@ebi.ac.uk> <4CA63D10.5030700@berkeley.edu> Message-ID: On 1 Oct 2010, at 20:57, Mitch Skinner wrote: > In JBrowse, the "schema" can vary by track; my assumption was that the set of populated attributes in an individual track would be pretty uniform. Some tracks might not use the "phase" field, for example, but if a given track used phase information, then I figured that almost all of the features in that track would populate that field. Indeed, and so long as your server/data file can determine in advance that phase is not used at all (as you say), it can omit it entirely. I'd say most of the time that's going to be perfectly possible. And to be honest I'm not wholly convinced that DAS has significantly less uniformity in the fields used within a data set, but it's worth us bearing in mind. > Also, javascript allows for array entries to be omitted entirely, like: > > [10000, 15000, , "foo"] > > JSON theoretically doesn't allow this; omitted entries become "undefined" in javascript, and the "official" JSON spec disallows "undefined" Interesting, I must admit my use of JSON is fairly limited so I wasn't aware of this possibility. It doesn't sound "nice", but when you're on the edge trying to squeeze out all the speed you can, it doesn't seem an issue. > > Also, depending on the use case, I wonder if the difference in (de)compression time between indexed and keyed JSON would matter. If you have your generated data handy still, I'd be curious to know what the difference is. I don't have the original files, but I do have the script to generate some more similar ones (attached). You could play around with whitespace too if you want to do some optimisation. -------------- next part -------------- A non-text attachment was scrubbed... Name: test-filesize.pl Type: text/x-perl-script Size: 1117 bytes Desc: not available URL: -------------- next part -------------- I'm sure there is a lot of existing study behind this very question. My expectation would be that the overhead of compression/decompression is going to be worth it for anything larger than 100 kb or so (probably even less), such is the huge difference it makes to the size of these files and bandwidth is usually the rate-limiting step. Up to now I have always considered that the chances are, if you're worried about speed and the size of your files, you probably need compression. Cheers, Andy From andy.jenkinson at ebi.ac.uk Wed Oct 6 16:27:38 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Wed, 6 Oct 2010 17:27:38 +0100 Subject: [DAS] 1.6 specification release Message-ID: Hi all, Just a quick note to say that since the last draft of 1.6, only clarifications to the text have been required and therefore the specification is considered fit for release. This is due to happen on Tuesday 19th October (I will be away from now until the 18th). On this date, although future revisions to the specification text are possible, the meaning and functionality will be frozen. The BioDAS website will be updated and an information page will be in place to help developers of clients and servers deal with any version compatibility issues arising from the switchover. In particular a list of clients and projected dates for 1.6 support. More details to follow in due course. Cheers, Andy From jw12 at sanger.ac.uk Thu Oct 7 11:08:09 2010 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 7 Oct 2010 12:08:09 +0100 Subject: [DAS] Removal of http://das.ensembl.org/das Message-ID: <0EC10E42-E483-4594-B55C-6A63FC6D8FCD@sanger.ac.uk> Hi All sources in the DAS registry which referred to http://das.ensembl.org/das/ now refer to http://das.sanger.ac.uk/das/ From now on please DO NOT refer to Sanger DAS sources with the old das.ensembl.org url as these DAS sources no longer show in ensembl. In the future only DAS sources served from the main ensembl/ensembl archive databases will be available from das.ensembl.org. Any problems please let me know. Sorry for any inconvenience caused but this had to be done due to technical reasons and as part of the Sanger DAS sources cleanup. Thanks Jonathan. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From andy.jenkinson at ebi.ac.uk Tue Oct 19 18:47:18 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Tue, 19 Oct 2010 19:47:18 +0100 Subject: [DAS] 1.6 specification release In-Reply-To: References: Message-ID: <5CE9BBBF-8638-49B0-B018-5DF627DB9676@ebi.ac.uk> The 1.6 specification is now online, and the BioDas homepage has been updated: http://www.biodas.org/ http://www.biodas.org/wiki/DAS1.6 http://www.biodas.org/wiki/DAS_1.6_Migration If you are a client or server developer, it'd be useful if you could fill in on the migration page any information you can regarding timescales for 1.6 support in your application. Please let me know if you spot any links/text on the website erroneously referring to the old spec (or just edit it yourself). Cheers, Andy On 6 Oct 2010, at 17:27, Andy Jenkinson wrote: > Hi all, > > Just a quick note to say that since the last draft of 1.6, only clarifications to the text have been required and therefore the specification is considered fit for release. This is due to happen on Tuesday 19th October (I will be away from now until the 18th). On this date, although future revisions to the specification text are possible, the meaning and functionality will be frozen. The BioDAS website will be updated and an information page will be in place to help developers of clients and servers deal with any version compatibility issues arising from the switchover. In particular a list of clients and projected dates for 1.6 support. > > More details to follow in due course. > > Cheers, > Andy > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Fri Oct 29 08:55:00 2010 From: dan.bolser at gmail.com (Dan Bolser) Date: Fri, 29 Oct 2010 10:55:00 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: <4CC71E8F.8020802@stanford.edu> References: <4CC71E8F.8020802@stanford.edu> Message-ID: Hi Bob, This is a nice idea. In my spare time I've been working on running a DAS server from a MediaWiki data source. (MediaWiki is the wiki that runs Wikipedia). In theory, you could have a comments track that was pulled in from the 'DASWiki', and clicking the details link would take you to the wiki page to edit the comment. Going back to the GBrowse, you would then see the updated data from the wiki page. It wouldn't be too hard to have a similar workflow for adding new comments. I saw a talk from some people at KeyGene who are doing something similar (loading DAS into a wiki and linking it to GBrowse to generate query reports for specific regions). You can see the talk here: SMWCon Fall 2010 / Sunday / Rudi van Bavel * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g ** http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv If anyone wants to take what I have done so far, my (sadly neglected) prototype is here: http://das.referata.com/wiki/Main_Page Kick me a few times and I'll get the underlying code into shape. A year or two back I did have a working prototype based on a MediaWiki ProServer adaptor. If you are interested in community annotation should think about attending this: Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: http://www.nettab.org/2010/ Cheers, Dan. On 26 October 2010 20:31, Bob Muller wrote: > We are working with a small research project that doesn't have funds for a full-scale database for features, so they want to know: > > How difficult would it be to add a comment section editable by users to gbrowse? ?When you click on a feature in gbrowse it shows a detail page for that feature, is there any way we could enable edits that could be written back to the underlying mysql database that gbrowse runs on? > > Their idea is to use the SeqFeature::Store database to hold "comments" entered through GBrowse as a way to annotate their genome. I'd be interested in whether this is possible in GBrowse 1.7 or in 2.x. > > ? --Bob Muller, TAIR (www.arabidopsis.org) > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in ?U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > Gmod-gbrowse mailing list > Gmod-gbrowse at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse > From gsalazar at cs.uct.ac.za Fri Oct 29 09:58:59 2010 From: gsalazar at cs.uct.ac.za (Gustavo Adolfo Salazar Orejuela) Date: Fri, 29 Oct 2010 11:58:59 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: Hello There, I have been working in an extension to DAS1.6 called writeback( http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow to create new versions of an existing feature. So, putting new notes into a feature is just a creation of a new version of the feature. The server implementation of the writeback extends MyDas (http://code.google.com/p/mydas/) and you can get a mydas+writebackk datasource by SVN ( https://mydas.googlecode.com/svn/writeback-datasource). We are close to release the plugin for Dasty3 to use the writeback server, however this first client is a protein client, and so far there is not a development for a genome client with DAS writeback capability, but I am quite interested in this. I hope to be able to announce the release of the writeback client in Dasty3 in a few days, so you will be able to see it working and see if this approach goes in the same direction of what you are looking for. Regards, Gustavo. 2010/10/29 Dan Bolser > Hi Bob, > > This is a nice idea. In my spare time I've been working on running a > DAS server from a MediaWiki data source. (MediaWiki is the wiki that > runs Wikipedia). > > In theory, you could have a comments track that was pulled in from the > 'DASWiki', and clicking the details link would take you to the wiki > page to edit the comment. Going back to the GBrowse, you would then > see the updated data from the wiki page. It wouldn't be too hard to > have a similar workflow for adding new comments. > > I saw a talk from some people at KeyGene who are doing something > similar (loading DAS into a wiki and linking it to GBrowse to generate > query reports for specific regions). You can see the talk here: > > SMWCon Fall 2010 / Sunday / Rudi van Bavel > * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program > ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g > ** > http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv > > > If anyone wants to take what I have done so far, my (sadly neglected) > prototype is here: > > http://das.referata.com/wiki/Main_Page > > > Kick me a few times and I'll get the underlying code into shape. A > year or two back I did have a working prototype based on a MediaWiki > ProServer adaptor. > > > If you are interested in community annotation should think about attending > this: > > Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: > http://www.nettab.org/2010/ > > > Cheers, > Dan. > > On 26 October 2010 20:31, Bob Muller wrote: > > We are working with a small research project that doesn't have funds for > a full-scale database for features, so they want to know: > > > > How difficult would it be to add a comment section editable by users to > gbrowse? When you click on a feature in gbrowse it shows a detail page for > that feature, is there any way we could enable edits that could be written > back to the underlying mysql database that gbrowse runs on? > > > > Their idea is to use the SeqFeature::Store database to hold "comments" > entered through GBrowse as a way to annotate their genome. I'd be interested > in whether this is possible in GBrowse 1.7 or in 2.x. > > > > --Bob Muller, TAIR (www.arabidopsis.org) > > > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > > Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > > Gmod-gbrowse mailing list > > Gmod-gbrowse at lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse > > > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das > -- Gustavo Adolfo Salazar Orejuela From andy.jenkinson at ebi.ac.uk Fri Oct 29 14:14:21 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Fri, 29 Oct 2010 15:14:21 +0100 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: <9B1B41B5-9200-4CD2-9667-45F342E57EF3@ebi.ac.uk> We also run a simple ProServer SourceAdaptor to pull in content from SNPedia, using the Perlwikipedia CPAN module. The solution of sending a user to a wiki sounds somewhat clunky to me, but would be relatively easy to set up I imagine. As Gustavo says, a full implementation of writeback in a production system from client through to backend is very close. Dasty3 (client) will be able to edit features directly in the interface, with the edits being stored in a DAS source separate from the original feature. Something similar could certainly be implemented for GBrowse, but would require development work. But as compensation, no development of the backend would be required. Cheers, Andy On 29 Oct 2010, at 09:55, Dan Bolser wrote: > Hi Bob, > > This is a nice idea. In my spare time I've been working on running a > DAS server from a MediaWiki data source. (MediaWiki is the wiki that > runs Wikipedia). > > In theory, you could have a comments track that was pulled in from the > 'DASWiki', and clicking the details link would take you to the wiki > page to edit the comment. Going back to the GBrowse, you would then > see the updated data from the wiki page. It wouldn't be too hard to > have a similar workflow for adding new comments. > > I saw a talk from some people at KeyGene who are doing something > similar (loading DAS into a wiki and linking it to GBrowse to generate > query reports for specific regions). You can see the talk here: > > SMWCon Fall 2010 / Sunday / Rudi van Bavel > * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program > ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g > ** http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv > > > If anyone wants to take what I have done so far, my (sadly neglected) > prototype is here: > > http://das.referata.com/wiki/Main_Page > > > Kick me a few times and I'll get the underlying code into shape. A > year or two back I did have a working prototype based on a MediaWiki > ProServer adaptor. > > > If you are interested in community annotation should think about attending this: > > Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: > http://www.nettab.org/2010/ > > > Cheers, > Dan. > > On 26 October 2010 20:31, Bob Muller wrote: >> We are working with a small research project that doesn't have funds for a full-scale database for features, so they want to know: >> >> How difficult would it be to add a comment section editable by users to gbrowse? When you click on a feature in gbrowse it shows a detail page for that feature, is there any way we could enable edits that could be written back to the underlying mysql database that gbrowse runs on? >> >> Their idea is to use the SeqFeature::Store database to hold "comments" entered through GBrowse as a way to annotate their genome. I'd be interested in whether this is possible in GBrowse 1.7 or in 2.x. >> >> --Bob Muller, TAIR (www.arabidopsis.org) >> >> ------------------------------------------------------------------------------ >> Nokia and AT&T present the 2010 Calling All Innovators-North America contest >> Create new apps & games for the Nokia N8 for consumers in U.S. and Canada >> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing >> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> http://p.sf.net/sfu/nokia-dev2dev >> _______________________________________________ >> Gmod-gbrowse mailing list >> Gmod-gbrowse at lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >> > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Fri Oct 29 15:28:04 2010 From: dan.bolser at gmail.com (Dan Bolser) Date: Fri, 29 Oct 2010 17:28:04 +0200 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: My thinking was that the DASWiki approach could also support the DAS writeback extension. On 29 October 2010 11:58, Gustavo Adolfo Salazar Orejuela wrote: > Hello There, > > I have been working in an extension to DAS1.6 called > writeback(http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow > to create new versions of an existing feature. So, putting new notes into a > feature is just a creation of a new version of the feature. The server > implementation of the writeback extends MyDas > (http://code.google.com/p/mydas/) and you can get a mydas+writebackk > datasource by SVN (https://mydas.googlecode.com/svn/writeback-datasource). > We are close to release the plugin for Dasty3 to use the writeback server, > however this first client is a protein client, and so far there is not a > development for a genome client with DAS writeback capability, but I am > quite interested in this. > I hope to be able to announce the release of the writeback client in Dasty3 > in a few days, so you will be able to see it working and see if this > approach goes in the same direction of what you are looking for. > > Regards, > > Gustavo. > > 2010/10/29 Dan Bolser >> >> Hi Bob, >> >> This is a nice idea. In my spare time I've been working on running a >> DAS server from a MediaWiki data source. (MediaWiki is the wiki that >> runs Wikipedia). >> >> In theory, you could have a comments track that was pulled in from the >> 'DASWiki', and clicking the details link would take you to the wiki >> page to edit the comment. Going back to the GBrowse, you would then >> see the updated data from the wiki page. It wouldn't be too hard to >> have a similar workflow for adding new comments. >> >> I saw a talk from some people at KeyGene who are doing something >> similar (loading DAS into a wiki and linking it to GBrowse to generate >> query reports for specific regions). You can see the talk here: >> >> SMWCon Fall 2010 / Sunday / Rudi van Bavel >> * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program >> ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g >> ** >> http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv >> >> >> If anyone wants to take what I have done so far, my (sadly neglected) >> prototype is here: >> >> http://das.referata.com/wiki/Main_Page >> >> >> Kick me a few times and I'll get the underlying code into shape. A >> year or two back I did have a working prototype based on a MediaWiki >> ProServer adaptor. >> >> >> If you are interested in community annotation should think about attending >> this: >> >> Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: >> http://www.nettab.org/2010/ >> >> >> Cheers, >> Dan. >> >> On 26 October 2010 20:31, Bob Muller wrote: >> > We are working with a small research project that doesn't have funds for >> > a full-scale database for features, so they want to know: >> > >> > How difficult would it be to add a comment section editable by users to >> > gbrowse? ?When you click on a feature in gbrowse it shows a detail page for >> > that feature, is there any way we could enable edits that could be written >> > back to the underlying mysql database that gbrowse runs on? >> > >> > Their idea is to use the SeqFeature::Store database to hold "comments" >> > entered through GBrowse as a way to annotate their genome. I'd be interested >> > in whether this is possible in GBrowse 1.7 or in 2.x. >> > >> > ? --Bob Muller, TAIR (www.arabidopsis.org) >> > >> > >> > ------------------------------------------------------------------------------ >> > Nokia and AT&T present the 2010 Calling All Innovators-North America >> > contest >> > Create new apps & games for the Nokia N8 for consumers in ?U.S. and >> > Canada >> > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >> > marketing >> > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >> > http://p.sf.net/sfu/nokia-dev2dev >> > _______________________________________________ >> > Gmod-gbrowse mailing list >> > Gmod-gbrowse at lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >> > >> >> _______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das > > > > -- > Gustavo Adolfo Salazar Orejuela > From andy.jenkinson at ebi.ac.uk Sat Oct 30 11:41:41 2010 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Sat, 30 Oct 2010 12:41:41 +0100 Subject: [DAS] [Gmod-gbrowse] Adding comments to a gbrowse database In-Reply-To: References: <4CC71E8F.8020802@stanford.edu> Message-ID: <6631318A-5425-417F-8220-4627B0116C36@ebi.ac.uk> Hi Dan, Sounds interesting. How do you see that working? Cheers, Andy On 29 Oct 2010, at 16:28, Dan Bolser wrote: > My thinking was that the DASWiki approach could also support the DAS > writeback extension. > > On 29 October 2010 11:58, Gustavo Adolfo Salazar Orejuela > wrote: >> Hello There, >> >> I have been working in an extension to DAS1.6 called >> writeback(http://www.biodas.org/wiki/DAS1.6E#DAS_writeback), this will allow >> to create new versions of an existing feature. So, putting new notes into a >> feature is just a creation of a new version of the feature. The server >> implementation of the writeback extends MyDas >> (http://code.google.com/p/mydas/) and you can get a mydas+writebackk >> datasource by SVN (https://mydas.googlecode.com/svn/writeback-datasource). >> We are close to release the plugin for Dasty3 to use the writeback server, >> however this first client is a protein client, and so far there is not a >> development for a genome client with DAS writeback capability, but I am >> quite interested in this. >> I hope to be able to announce the release of the writeback client in Dasty3 >> in a few days, so you will be able to see it working and see if this >> approach goes in the same direction of what you are looking for. >> >> Regards, >> >> Gustavo. >> >> 2010/10/29 Dan Bolser >>> >>> Hi Bob, >>> >>> This is a nice idea. In my spare time I've been working on running a >>> DAS server from a MediaWiki data source. (MediaWiki is the wiki that >>> runs Wikipedia). >>> >>> In theory, you could have a comments track that was pulled in from the >>> 'DASWiki', and clicking the details link would take you to the wiki >>> page to edit the comment. Going back to the GBrowse, you would then >>> see the updated data from the wiki page. It wouldn't be too hard to >>> have a similar workflow for adding new comments. >>> >>> I saw a talk from some people at KeyGene who are doing something >>> similar (loading DAS into a wiki and linking it to GBrowse to generate >>> query reports for specific regions). You can see the talk here: >>> >>> SMWCon Fall 2010 / Sunday / Rudi van Bavel >>> * http://semantic-mediawiki.org/wiki/SMWCon_Fall_2010#Program >>> ** https://docs.google.com/present/view?id=dg56kh6m_344gh2dg72g >>> ** >>> http://srv-hrl-32.web.pwo.ou.nl//informatica/SMWCon/Van%20Bavel%20bulk%20loading.wmv >>> >>> >>> If anyone wants to take what I have done so far, my (sadly neglected) >>> prototype is here: >>> >>> http://das.referata.com/wiki/Main_Page >>> >>> >>> Kick me a few times and I'll get the underlying code into shape. A >>> year or two back I did have a working prototype based on a MediaWiki >>> ProServer adaptor. >>> >>> >>> If you are interested in community annotation should think about attending >>> this: >>> >>> Joint NETTAB 2010 and BBCC 2010 workshops focused on Biological Wikis: >>> http://www.nettab.org/2010/ >>> >>> >>> Cheers, >>> Dan. >>> >>> On 26 October 2010 20:31, Bob Muller wrote: >>>> We are working with a small research project that doesn't have funds for >>>> a full-scale database for features, so they want to know: >>>> >>>> How difficult would it be to add a comment section editable by users to >>>> gbrowse? When you click on a feature in gbrowse it shows a detail page for >>>> that feature, is there any way we could enable edits that could be written >>>> back to the underlying mysql database that gbrowse runs on? >>>> >>>> Their idea is to use the SeqFeature::Store database to hold "comments" >>>> entered through GBrowse as a way to annotate their genome. I'd be interested >>>> in whether this is possible in GBrowse 1.7 or in 2.x. >>>> >>>> --Bob Muller, TAIR (www.arabidopsis.org) >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Nokia and AT&T present the 2010 Calling All Innovators-North America >>>> contest >>>> Create new apps & games for the Nokia N8 for consumers in U.S. and >>>> Canada >>>> $10 million total in prizes - $4M cash, 500 devices, nearly $6M in >>>> marketing >>>> Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store >>>> http://p.sf.net/sfu/nokia-dev2dev >>>> _______________________________________________ >>>> Gmod-gbrowse mailing list >>>> Gmod-gbrowse at lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse >>>> >>> >>> _______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das >> >> >> >> -- >> Gustavo Adolfo Salazar Orejuela >> > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das