From jw12 at sanger.ac.uk Thu Jan 5 04:57:05 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 5 Jan 2012 09:57:05 +0000 Subject: [DAS] workshop email Message-ID: Hi DAS people. I intend to circulate to other email lists the DAS workshop email just sent. If anyone has suggestions as to other email lists to post it to or would like to forward this email on to them (saves me registering on lots of lists :) ) then please do (but let me know so I don't duplicate posts). List I intend to use are: gmod biojava ensembl bioperl Sanger/EBI campus We could do with it being circulated around less developer focused lists e.g. plant focused user groups? More general biology lists? Suggestions? Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Thu Jan 5 04:49:02 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 5 Jan 2012 09:49:02 +0000 Subject: [DAS] Registrations for DAS Workshop 2012 Message-ID: <69FA2B25-F15E-4361-BBAC-E6DC3975F435@sanger.ac.uk> DAS is currently being used to share annotations on genomes, protein alignments, structural and interaction information. If you are interested in sharing biological information the DAS workshop below may be of interest to you. Learn of and contribute to current developments in DAS such as: DAS in the cloud, DAS for Genotype Data, DAS searching, DAS for collaborative annotation projects, DAS alternative formats. Registration is open for the 2012 DAS workshop (27-29 February) at the Genome Campus, Hinxton UK. If you are interested in attending, please find out more by going to http://www.ebi.ac.uk/training/onsite/120227_DAS.html and register via the web link at the bottom of the page. This workshop will cater for novice to expert DAS users as each day is optional. Please register early as places will be limited. Registration closes 10 February 2012 - 12:00. If you are interested in giving a 15 minute talk on the second day please email Jonathan Warren using jonathan.warren at sanger.ac.uk Many thanks The Sanger/EBI DAS team. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Wed Jan 11 09:20:54 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Wed, 11 Jan 2012 14:20:54 +0000 Subject: [DAS] Workshop 2012 format change Message-ID: <18968EB6-4CE3-489E-88E6-31D206AFFC7C@sanger.ac.uk> After some discussions and partly due to a lack of volunteers for talks we have decided to modify the DAS workshop format this year. The first day as per previous years will be tutorials and talks for new users of DAS. The second and third day will be for DAS developers as a hackathon to facilitate the implementation and updating of clients, servers and libraries to implement the latest 1.6E specifications. Thus for this year we are dispensing with the second day of talks which will hopefully be back in some form next year. The workshop registration page and registrations will be updated to reflect this change soon. Any comments or suggestions please let us know. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Thu Jan 12 08:21:14 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 12 Jan 2012 13:21:14 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps Message-ID: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Hi I've put some instructions for people with little or no technical ability and no access to IT personnel or servers, to be able to publish there genotype data from companies such as 23andme etc as a DAS source on the amazon cloud. http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ All someone needs is a tab delimited text file from one of these companies, a credit card and an internet connection. The instructions show them how to set up and deploy their server to the cloud in 6 easy steps and then view the data in Ensembl. Example server can be accessed from here http://mychoiceofname.elasticbeanstalk.com/das/person1 Any comments suggestions welcomed. Cheers Jonathan. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From dan.bolser at gmail.com Thu Jan 12 08:43:59 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 13:43:59 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Message-ID: That's great! Just last night I was looking for something that would fulfill this requirement. One suggestion would be for authentication like I get with my Google, Flicker, Twitter, Facebook, etc. accounts. (Is this an open protocol for authentication, or did they each re-invent this wheel?) i.e. I want to be able to specifically grant access to my data by a known third party. e.g. GeneticGuru wants access to your GenotypeArchive account, allow or deny? Do any existing personal genotype archives have this kind of 'genotype server' service, or is DAS the first system to provide this functionality? Do you plan to build it into a dedicated service somewhere? Cheers, Dan. On 12 January 2012 13:21, Jonathan Warren wrote: > Hi > > I've put some instructions for people with little or no technical ability > and no access to IT personnel or servers, to be able to publish there > genotype data from companies such as 23andme etc as a DAS source on the > amazon cloud. > > http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ > > All someone needs is a tab delimited text file from one of these companies, > a credit card and an internet connection. The instructions show them how to > set up and deploy their server to the cloud in 6 easy steps and then view > the data in Ensembl. > > Example server can be accessed from here > http://mychoiceofname.elasticbeanstalk.com/das/person1 > > Any comments suggestions welcomed. > > Cheers > > Jonathan. > > Jonathan Warren > Senior Developer and DAS coordinator > blog: http://biodasman.wordpress.com/ > jw12 at sanger.ac.uk > Ext: 2314 > Telephone: 01223 492314 > > > > > > > > > > > -- > The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a > charity registered in England with number 1021457 and acompany registered in > England with number 2742969, whose registeredoffice is 215 Euston Road, > London, NW1 2BE._______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From jw12 at sanger.ac.uk Thu Jan 12 10:31:52 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 12 Jan 2012 15:31:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Message-ID: <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> On 12 Jan 2012, at 13:43, Dan Bolser wrote: Hi Dan Thanks for the feedback. > That's great! > > Just last night I was looking for something that would fulfill this > requirement. > > One suggestion would be for authentication like I get with my Google, > Flicker, Twitter, Facebook, etc. accounts. (Is this an open protocol > for authentication, or did they each re-invent this wheel?) > > i.e. I want to be able to specifically grant access to my data by a > known third party. > > e.g. GeneticGuru wants access to your GenotypeArchive account, allow > or deny? > For the situation where someone only wants themselves to access there own data for the moment they can just turn on their DAS server when they want to use it and not advertise their url. But if there is demand for people to be able to specify who can use it I can add some security to this. We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. The registry already supports this for the web service and I believe Andy J has implemented it in proserver as well. MyDAS if it doesn't already have it, can be easily modified to do so I think and it's on my list to do this for a writeback instance I'm setting up for genomic data anyway. However the big fly in the ointment is that I'm pretty sure no clients support it at the moment however. I'm sure ensembl doesn't and I don't think Dalliance does, GBrowse doesn't? I think maybe Web Apollo does, but I haven't heard anything from those guys in ages? I hope I'm wrong on this though. Maybe Karyodas can implement it quickly???? http://mykaryoview.com > Do any existing personal genotype archives have this kind of 'genotype > server' service, or is DAS the first system to provide this > functionality? Do you plan to build it into a dedicated service > somewhere? I'm in discussions about this with the powers that be at the moment. Problems would include massive memory usage if implemented this way and also legal issues around managing other peoples genomic data. Which is why the personal virtual machine in the cloud is something that is quite an elegant solution at the moment :) I would like the process for people to be even easier though and for setting up DAS sources in general. > > > Cheers, > Dan. > > On 12 January 2012 13:21, Jonathan Warren wrote: >> Hi >> >> I've put some instructions for people with little or no technical >> ability >> and no access to IT personnel or servers, to be able to publish there >> genotype data from companies such as 23andme etc as a DAS source on >> the >> amazon cloud. >> >> http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ >> >> All someone needs is a tab delimited text file from one of these >> companies, >> a credit card and an internet connection. The instructions show >> them how to >> set up and deploy their server to the cloud in 6 easy steps and >> then view >> the data in Ensembl. >> >> Example server can be accessed from here >> http://mychoiceofname.elasticbeanstalk.com/das/person1 >> >> Any comments suggestions welcomed. >> >> Cheers >> >> Jonathan. >> >> Jonathan Warren >> Senior Developer and DAS coordinator >> blog: http://biodasman.wordpress.com/ >> jw12 at sanger.ac.uk >> Ext: 2314 >> Telephone: 01223 492314 >> >> >> >> >> >> >> >> >> >> >> -- >> The Wellcome Trust Sanger Institute is operated by Genome >> ResearchLimited, a >> charity registered in England with number 1021457 and acompany >> registered in >> England with number 2742969, whose registeredoffice is 215 Euston >> Road, >> London, NW1 2BE._______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From andy.jenkinson at ebi.ac.uk Thu Jan 12 12:07:50 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Thu, 12 Jan 2012 17:07:50 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> Message-ID: <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Great stuff Jon! On 12 Jan 2012, at 15:31, Jonathan Warren wrote: > > On 12 Jan 2012, at 13:43, Dan Bolser wrote: > >> I want to be able to specifically grant access to my data by a known third party. > > We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. Standard HTTP authentication by contrast is much easier to implement and is a sort of "easy win" for the interim. > The registry already supports this for the web service and I believe Andy J has implemented it in proserver as well. MyDAS if it doesn't already have it, can be easily modified to do so I think and it's on my list to do this for a writeback instance I'm setting up for genomic data anyway. > > However the big fly in the ointment is that I'm pretty sure no clients support it at the moment however. I'm sure ensembl doesn't and I don't think Dalliance does, GBrowse doesn't? I think maybe Web Apollo does, but I haven't heard anything from those guys in ages? I hope I'm wrong on this though. Maybe Karyodas can implement it quickly???? http://mykaryoview.com Actually Dalliance does support HTTP authentication because, being pure Javascript, it uses the browser's implementation. The development branch of ProServer includes support for HTTP authentication and SSL encryption. I'm fairly sure the authentication (basic and digest) work well, but I think the encryption could use some testing. The main barrier though is as Jonathan says that no other clients support it, it would be easier to debug using these rather than using browsers directly. Obviously what we really want is for Ensembl to implement it. I'm not sure what it would take for myKaryoview and Dasty, they are javascript clients but I think they use a server-side proxy so it might be hard/impossible for them to use the browser's cache of passwords. That means storing them locally, which means some sort of login system, which means extra security considerations... etc. I think Jonathan's personal genomics server is a clear use case for authentication, and I would also seek to incorporate it into EasyDAS (Which uses ProServer). We need major clients to adopt it, and it is of course the classic DAS chicken and egg situation. Cheers, Andy > >> Do any existing personal genotype archives have this kind of 'genotype >> server' service, or is DAS the first system to provide this >> functionality? Do you plan to build it into a dedicated service >> somewhere? > I'm in discussions about this with the powers that be at the moment. Problems would include massive memory usage if implemented this way and also legal issues around managing other peoples genomic data. Which is why the personal virtual machine in the cloud is something that is quite an elegant solution at the moment :) I would like the process for people to be even easier though and for setting up DAS sources in general. > >> >> >> Cheers, >> Dan. >> >> On 12 January 2012 13:21, Jonathan Warren wrote: >>> Hi >>> >>> I've put some instructions for people with little or no technical ability >>> and no access to IT personnel or servers, to be able to publish there >>> genotype data from companies such as 23andme etc as a DAS source on the >>> amazon cloud. >>> >>> http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ >>> >>> All someone needs is a tab delimited text file from one of these companies, >>> a credit card and an internet connection. The instructions show them how to >>> set up and deploy their server to the cloud in 6 easy steps and then view >>> the data in Ensembl. >>> >>> Example server can be accessed from here >>> http://mychoiceofname.elasticbeanstalk.com/das/person1 >>> >>> Any comments suggestions welcomed. >>> >>> Cheers >>> >>> Jonathan. >>> >>> Jonathan Warren >>> Senior Developer and DAS coordinator >>> blog: http://biodasman.wordpress.com/ >>> jw12 at sanger.ac.uk >>> Ext: 2314 >>> Telephone: 01223 492314 >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a >>> charity registered in England with number 1021457 and acompany registered in >>> England with number 2742969, whose registeredoffice is 215 Euston Road, >>> London, NW1 2BE._______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das > > Jonathan Warren > Senior Developer and DAS coordinator > blog: http://biodasman.wordpress.com/ > jw12 at sanger.ac.uk > Ext: 2314 > Telephone: 01223 492314 > > > > > > > > > > > -- > The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a charity registered in England with number 1021457 and acompany registered in England with number 2742969, whose registeredoffice is 215 Euston Road, London, NW1 2BE._______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Thu Jan 12 13:08:01 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 18:08:01 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Message-ID: On 12 January 2012 17:07, Andy Jenkinson wrote: > Great stuff Jon! > > On 12 Jan 2012, at 15:31, Jonathan Warren wrote: >> >> On 12 Jan 2012, at 13:43, Dan Bolser wrote: >> >>> I want to be able to specifically grant access to my data by a known third party. >> >> We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. > > I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. AFAIK, using something like the above, you authenticate with the client using OpenID, and the client is authenticated to access your data via OAuth. You can then build your client to allow various levels of sharing with other users in the system, as with FB. Would building OAuth into Proserver, then identifying with OpenID be a way round the 'technical reasons' you described above? Or is it just running in circles? Cheers, Dan. From dan.bolser at gmail.com Thu Jan 12 13:09:48 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 18:09:48 +0000 Subject: [DAS] DAS2GFF3? Message-ID: Hi, Given a set of features in DAS format, what is the easiest way to generate GFF3 format? Can most DAS server implementation be persuaded to serve GFF3 in response to DAS feature requests? Cheers, Dan. From andy.jenkinson at ebi.ac.uk Thu Jan 12 19:42:50 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Fri, 13 Jan 2012 00:42:50 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Message-ID: <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> On 12 Jan 2012, at 18:08, Dan Bolser wrote: > On 12 January 2012 17:07, Andy Jenkinson wrote: >> Great stuff Jon! >> >> On 12 Jan 2012, at 15:31, Jonathan Warren wrote: >>> >>> On 12 Jan 2012, at 13:43, Dan Bolser wrote: >>> >>>> I want to be able to specifically grant access to my data by a known third party. >>> >>> We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. >> >> I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. > > AFAIK, using something like the above, you authenticate with the > client using OpenID, and the client is authenticated to access your > data via OAuth. You can then build your client to allow various levels > of sharing with other users in the system, as with FB. > > Would building OAuth into Proserver, then identifying with OpenID be a > way round the 'technical reasons' you described above? Or is it just > running in circles? Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still setting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had the resources to do it. Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. Cheers, Andy From jprocter at compbio.dundee.ac.uk Fri Jan 13 08:04:43 2012 From: jprocter at compbio.dundee.ac.uk (Jim Procter) Date: Fri, 13 Jan 2012 13:04:43 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> Message-ID: <4F102BEB.1020101@compbio.dundee.ac.uk> Hi. I see the dreaded secure DAS session topic as risen its head again. On 13/01/2012 00:42, Andy Jenkinson wrote: > Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. > It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still set! > ting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. Yes. security isn't cheap it seems :) > I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had ! > the resources to do it. So the problem here with OAuth is that pure browser clients need a secure store for authenticating a user's access to a particular resource ? I don't think there is any way around that either, and I think the onus to provide this is the hosting page of the client - i.e. dalliance.org would need to be recognised as a peer on a secure DAS OAuth network (using the servers own keys). Then, users wanting access to secure data would log in to the DAS source independently via a secure session on dalliance.org, which would then allow dalliance to browse the data from the secure server. If someone wants to set up a Dalliance browser in their own OAuth trust network, then they would need to have their own Dalliance hosting server and make it known to the other peers accordingly. > Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. Jim. From andy.jenkinson at ebi.ac.uk Fri Jan 13 20:03:40 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Sat, 14 Jan 2012 01:03:40 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F102BEB.1020101@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> Message-ID: <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> On 13 Jan 2012, at 13:04, Jim Procter wrote: > Hi. I see the dreaded secure DAS session topic as risen its head again. > > On 13/01/2012 00:42, Andy Jenkinson wrote: >> Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. > But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. Not sure I follow. The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). >> It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still s! > et! >> ting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. > Yes. security isn't cheap it seems :) >> I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had ! >> the resources to do it. > So the problem here with OAuth is that pure browser clients need a secure store for authenticating a user's access to a particular resource ? I don't think there is any way around that either, and I think the onus to provide this is the hosting page of the client - i.e. dalliance.org would need to be recognised as a peer on a secure DAS OAuth network (using the servers own keys). Then, users wanting access to secure data would log in to the DAS source independently via a secure session on dalliance.org, which would then allow dalliance to browse the data from the secure server. If someone wants to set up a Dalliance browser in their own OAuth trust network, then they would need to have their own Dalliance hosting server and make it known to the other peers accordingly. Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. However, I just saw this, which is a potential solution (for OAuth at least): http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. >> Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. > I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). > Jim. > > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From jprocter at compbio.dundee.ac.uk Mon Jan 16 11:44:55 2012 From: jprocter at compbio.dundee.ac.uk (Jim Procter) Date: Mon, 16 Jan 2012 16:44:55 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> Message-ID: <4F145407.4000900@compbio.dundee.ac.uk> Hi Andy, and anyone else still reading :) On 14/01/2012 01:03, Andy Jenkinson wrote: > On 13 Jan 2012, at 13:04, Jim Procter wrote: > >> Hi. I see the dreaded secure DAS session topic as risen its head again. >> >> On 13/01/2012 00:42, Andy Jenkinson wrote: >>> Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. >> But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. > Not sure I follow. Bother! am I feeding troll(s) again ? ;) > The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). I'm not sure how different this is from the 3D security challenge from your bank/credit card company that opens in an iframe after you put your credit card details in to a secure shopping site and hit the confirm button. Authentication still has to happen in order for a transaction (in our case one involving a DAS request/response session) between the DAS server and the Ensembl server, before it renders the result in the page sent to your browser. As for dalliance and secure servers: > Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. it won't actually be happening that way - take a look at this: http://www.biodalliance.org/cors.html > However, I just saw this, which is a potential solution (for OAuth at least): > http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ > I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. Interesting angle. I'm not actually sure what happens if the page from a server allowed to contact another server via CORS has been edited after it arrived in the browser sandbox - but I don't think it is really that much of an issue either way. There are two situations worth considering: 1. an honest user wants to access secure data. In which case, they must authenticate via the dalliance host page with servers providing secure data in order to grant dalliance access via an authenticated cross-origin request from a specified server on the trusted CORS list. 2. a black hat attempts to gain access to secure data. In which case they must fake an authentication. We then have to rely on the authentication framework detecting cracking attempts. The case when a (probably) honest user with valid credentials misuses the cross-origin request by messing with the javascript on the page is no different to having a user in a secure intranet who brings a virus in via a physical device or over a VPN. Since DAS is rather limited in its scope for exploits, I don't think there will be a problem with someone fiddling with the javascript, assuming, of course, the servers hosting the authenticated sources actually secured their server properly and have un-DOSable data providers at the back end... >>> Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. >> I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. > I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). > understood, and I agree, with the caveat that I'm not even sure how familiar we all are with HTTP(s) auth styles, and personally, would appreciate it if there was a practical session or tutorial on the ins and outs of authenticated DAS for those who need to do it. This could cover how different styles of authenticated DAS sources work with the existing clients (e.g. do I really have to authenticate against every secure das source every time a request is made?!). It could also involve a tutorial on setting up a secure DAS source using various schemes. For example, a tutorial on SSO systems such as Shibboleth2 as used by the UK access management federation could be useful if people needs to give specific UK academics access to a DAS source. Of course - nothing in this email constitutes me volunteering to give such a tutorial .... Jim. From mc at manuelcorpas.com Mon Jan 16 12:10:29 2012 From: mc at manuelcorpas.com (Manuel Corpas) Date: Mon, 16 Jan 2012 17:10:29 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F145407.4000900@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Hi guys, why not just trying to do something *really* easy that could be implemented in a couple of weeks rather than trying to engineer the perfect solution all at once? We don't really know how much demand there is for authenticated DAS sources, so we may end up spending a lot of effort for something that might be underused. The truth is that currently all DAS sources are public. Some "private sources" available are those provided by easyDAS with its long "cryptic" urls. That alone I think is already something. Any improvement on the ability to keep sources private, whatever small, would be a step in the right direction. My two cents. Manuel Manuel Corpas, PhD Tel:? ? ? +44.122349.2372 Web: ? ?http://manuelcorpas.com/about/ Twitter: @manuelcorpas On 16 January 2012 16:44, Jim Procter wrote: > Hi Andy, and anyone else still reading :) > > > On 14/01/2012 01:03, Andy Jenkinson wrote: >> >> On 13 Jan 2012, at 13:04, Jim Procter wrote: >> >>> Hi. I see the dreaded secure DAS session topic as risen its head again. >>> >>> On 13/01/2012 00:42, Andy Jenkinson wrote: >>>> >>>> Originally I wanted to use a combination of OpenID and OAuth as an >>>> end-to-end solution. However, OpenID is based around the expectation that >>>> you are authenticating with a website using a browser - the protocol uses >>>> HTTP redirects, and OpenID providers have to have some way of telling you >>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server >>>> that needs to check that you are who you say you are, not just the client. >>>> For a client like Ensembl, your browser simply never communicates with the >>>> DAS server so the DAS server can't get you to authenticate with the OpenID >>>> provider. >>> >>> But Ensembl *does* need to know who you are in order to request data that >>> you are allowed access to. In the OAuth model, you would have to allow >>> Ensembl to access privileged data from the third party DAS server, and that >>> would be achieved by the Ensembl browser presenting you with an OpenID login >>> and redirect to subsequent access control pages from the third party server. >> >> Not sure I follow. > > Bother! am I feeding troll(s) again ? ;) > >> The problem with the above model isn't with Ensembl it's with the DAS >> servers. DAS servers can't verify the user that is sitting at his machine >> (i.e. use OpenID) because they don't communicate with his machine. They only >> communicate, and can thus verify, the client (i.e. using OAuth). It would be >> better if the DAS server had some way to check the user directly (the client >> would be like a proxy for the user) but it is not possible with OpenID under >> Ensembl's current architecture (or at least it wasn't when I was >> investigating). So that means you have to do one of the below options >> instead (including having only Ensembl do the authentication). > > I'm not sure how different this is from the 3D security challenge from your > bank/credit card company that opens in an iframe after you put your credit > card details in to a secure shopping site and hit the confirm button. > Authentication still has to happen in order for a transaction (in our case > one involving a DAS request/response session) between the DAS server and the > Ensembl server, before it renders the result in the page sent to your > browser. > > As for dalliance and secure servers: > >> Basically yes, I would have thought there'd have to be some server-side >> Dalliance script doing most of the work, with it hopefully being possible to >> pass the signed token to the browser so that the user's machine is still the >> one issuing the DAS request. > > it won't actually be happening that way - take a look at this: > http://www.biodalliance.org/cors.html > >> However, I just saw this, which is a potential solution (for OAuth at >> least): >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >> I'm not sure it helps however, as we need to ensure that Dalliance will >> only issue an authorised request to the DAS server if it has proof that the >> user has been authenticated and is on the access control list. With JS being >> editable on the client side, that check cannot be in JS. > > Interesting angle. I'm not actually sure what happens if the page from a > server allowed to contact another server via CORS has been edited after it > arrived in the browser sandbox - but I don't think it is really that much of > an issue either way. > > There are two situations worth considering: > 1. an honest user wants to access secure data. In which case, they must > authenticate via the dalliance host page with servers providing secure data > in order to grant dalliance access via an authenticated cross-origin request > from a specified server on the trusted CORS list. > 2. a black hat attempts to gain access to secure data. In which case they > must fake an authentication. We then have to rely on the authentication > framework detecting cracking attempts. > > The case when a (probably) honest user with valid credentials misuses the > cross-origin request by messing with the javascript on the page is no > different to having a user in a secure intranet who brings a virus in via a > physical device or over a VPN. Since DAS is rather limited in its scope for > exploits, I don't think there will be a problem with someone fiddling with > the javascript, assuming, of course, the servers hosting the authenticated > sources actually secured their server properly and have un-DOSable data > providers at the back end... > >>>> Lastly, neither solution works for daemon-style clients (e.g. >>>> command-line analysis applications where the user is not present), again >>>> because they can't use OpenID. The catch-all solution is to use X.509 >>>> certificates (public/private key cryptography) but it is heavyweight and >>>> probably too complicated to provide a good user experience. Truth be told, >>>> it has proved difficult to discuss this topic amongst the community because >>>> it gets technically very complex. >>> >>> I'd be very much in favour of people who *need* to achieve this spending >>> some time during the developer days with an invited expert (with special >>> anti-trolling skills to counter folk like me) in a closed session, in order >>> to identify components that would enable both browser and non-browser based >>> clients to work with OAuth, and set up a trial OAuth DAS source network for >>> testing. There are libraries that support OAuth (http://oauth.net/code/) >>> both for providers and consumers, but DAS client libraries will need >>> extension to allow secure negotiation and signed DAS HTTP interaction, and >>> their APIs will need additional parameters/methods to allow session >>> management. >> >> I have no objections to participating in principle, but I remain to >> convinced it is worth the effort (i.e. that HTTP auth is insufficient and >> that OAuth would be adopted). >> > understood, and I agree, with the caveat that I'm not even sure how familiar > we all are with HTTP(s) auth styles, and personally, would appreciate it if > there was a practical session or tutorial on the ins and outs of > authenticated DAS for those who need to do it. This could cover how > different styles of authenticated DAS sources work with the existing clients > (e.g. do I really have to authenticate against every secure das source every > time a request is made?!). It could also involve a tutorial on setting up a > secure DAS source using various schemes. For example, a tutorial on SSO > systems such as Shibboleth2 as used by the UK access management federation > could be useful if people needs to give specific UK academics access to a > DAS source. > > Of course - nothing in this email constitutes me volunteering to give such a > tutorial .... > > Jim. > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From bernatgel at gmail.com Mon Jan 16 12:20:55 2012 From: bernatgel at gmail.com (Bernat Gel) Date: Mon, 16 Jan 2012 18:20:55 +0100 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Hi all, I agree with Manuel (and I think Andy) that without a real demand for private DAS sources implementing some complex authentication scheme might be an overkill. However, I think that a key thing with security is that if it's done, it has to be weel done. Somehow, a false sense of privacy (such as the cryptic URLs in easyDAS) might be a bad thing if the user is not aware of its weaknesses and end up unkwowingly sharing data deemed to remain private. So, unless a DAS sourceis really private and secure, it should be made clear that it is public (although maybe difficult to find). Bernat On Mon, Jan 16, 2012 at 18:10, Manuel Corpas wrote: > Hi guys, > > why not just trying to do something *really* easy that could be > implemented in a couple of weeks rather than trying to engineer the > perfect solution all at once? We don't really know how much demand > there is for authenticated DAS sources, so we may end up spending a > lot of effort for something that might be underused. > > The truth is that currently all DAS sources are public. Some "private > sources" available are those provided by easyDAS with its long > "cryptic" urls. That alone I think is already something. Any > improvement on the ability to keep sources private, whatever small, > would be a step in the right direction. > > My two cents. > > Manuel > > Manuel Corpas, PhD > Tel: +44.122349.2372 > Web: http://manuelcorpas.com/about/ > Twitter: @manuelcorpas > > > > On 16 January 2012 16:44, Jim Procter > wrote: > > Hi Andy, and anyone else still reading :) > > > > > > On 14/01/2012 01:03, Andy Jenkinson wrote: > >> > >> On 13 Jan 2012, at 13:04, Jim Procter wrote: > >> > >>> Hi. I see the dreaded secure DAS session topic as risen its head again. > >>> > >>> On 13/01/2012 00:42, Andy Jenkinson wrote: > >>>> > >>>> Originally I wanted to use a combination of OpenID and OAuth as an > >>>> end-to-end solution. However, OpenID is based around the expectation > that > >>>> you are authenticating with a website using a browser - the protocol > uses > >>>> HTTP redirects, and OpenID providers have to have some way of telling > you > >>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS > server > >>>> that needs to check that you are who you say you are, not just the > client. > >>>> For a client like Ensembl, your browser simply never communicates > with the > >>>> DAS server so the DAS server can't get you to authenticate with the > OpenID > >>>> provider. > >>> > >>> But Ensembl *does* need to know who you are in order to request data > that > >>> you are allowed access to. In the OAuth model, you would have to allow > >>> Ensembl to access privileged data from the third party DAS server, and > that > >>> would be achieved by the Ensembl browser presenting you with an OpenID > login > >>> and redirect to subsequent access control pages from the third party > server. > >> > >> Not sure I follow. > > > > Bother! am I feeding troll(s) again ? ;) > > > >> The problem with the above model isn't with Ensembl it's with the DAS > >> servers. DAS servers can't verify the user that is sitting at his > machine > >> (i.e. use OpenID) because they don't communicate with his machine. They > only > >> communicate, and can thus verify, the client (i.e. using OAuth). It > would be > >> better if the DAS server had some way to check the user directly (the > client > >> would be like a proxy for the user) but it is not possible with OpenID > under > >> Ensembl's current architecture (or at least it wasn't when I was > >> investigating). So that means you have to do one of the below options > >> instead (including having only Ensembl do the authentication). > > > > I'm not sure how different this is from the 3D security challenge from > your > > bank/credit card company that opens in an iframe after you put your > credit > > card details in to a secure shopping site and hit the confirm button. > > Authentication still has to happen in order for a transaction (in our > case > > one involving a DAS request/response session) between the DAS server and > the > > Ensembl server, before it renders the result in the page sent to your > > browser. > > > > As for dalliance and secure servers: > > > >> Basically yes, I would have thought there'd have to be some server-side > >> Dalliance script doing most of the work, with it hopefully being > possible to > >> pass the signed token to the browser so that the user's machine is > still the > >> one issuing the DAS request. > > > > it won't actually be happening that way - take a look at this: > > http://www.biodalliance.org/cors.html > > > >> However, I just saw this, which is a potential solution (for OAuth at > >> least): > >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ > >> I'm not sure it helps however, as we need to ensure that Dalliance will > >> only issue an authorised request to the DAS server if it has proof that > the > >> user has been authenticated and is on the access control list. With JS > being > >> editable on the client side, that check cannot be in JS. > > > > Interesting angle. I'm not actually sure what happens if the page from a > > server allowed to contact another server via CORS has been edited after > it > > arrived in the browser sandbox - but I don't think it is really that > much of > > an issue either way. > > > > There are two situations worth considering: > > 1. an honest user wants to access secure data. In which case, they must > > authenticate via the dalliance host page with servers providing secure > data > > in order to grant dalliance access via an authenticated cross-origin > request > > from a specified server on the trusted CORS list. > > 2. a black hat attempts to gain access to secure data. In which case they > > must fake an authentication. We then have to rely on the authentication > > framework detecting cracking attempts. > > > > The case when a (probably) honest user with valid credentials misuses the > > cross-origin request by messing with the javascript on the page is no > > different to having a user in a secure intranet who brings a virus in > via a > > physical device or over a VPN. Since DAS is rather limited in its scope > for > > exploits, I don't think there will be a problem with someone fiddling > with > > the javascript, assuming, of course, the servers hosting the > authenticated > > sources actually secured their server properly and have un-DOSable data > > providers at the back end... > > > >>>> Lastly, neither solution works for daemon-style clients (e.g. > >>>> command-line analysis applications where the user is not present), > again > >>>> because they can't use OpenID. The catch-all solution is to use X.509 > >>>> certificates (public/private key cryptography) but it is heavyweight > and > >>>> probably too complicated to provide a good user experience. Truth be > told, > >>>> it has proved difficult to discuss this topic amongst the community > because > >>>> it gets technically very complex. > >>> > >>> I'd be very much in favour of people who *need* to achieve this > spending > >>> some time during the developer days with an invited expert (with > special > >>> anti-trolling skills to counter folk like me) in a closed session, in > order > >>> to identify components that would enable both browser and non-browser > based > >>> clients to work with OAuth, and set up a trial OAuth DAS source > network for > >>> testing. There are libraries that support OAuth ( > http://oauth.net/code/) > >>> both for providers and consumers, but DAS client libraries will need > >>> extension to allow secure negotiation and signed DAS HTTP interaction, > and > >>> their APIs will need additional parameters/methods to allow session > >>> management. > >> > >> I have no objections to participating in principle, but I remain to > >> convinced it is worth the effort (i.e. that HTTP auth is insufficient > and > >> that OAuth would be adopted). > >> > > understood, and I agree, with the caveat that I'm not even sure how > familiar > > we all are with HTTP(s) auth styles, and personally, would appreciate it > if > > there was a practical session or tutorial on the ins and outs of > > authenticated DAS for those who need to do it. This could cover how > > different styles of authenticated DAS sources work with the existing > clients > > (e.g. do I really have to authenticate against every secure das source > every > > time a request is made?!). It could also involve a tutorial on setting > up a > > secure DAS source using various schemes. For example, a tutorial on SSO > > systems such as Shibboleth2 as used by the UK access management > federation > > could be useful if people needs to give specific UK academics access to a > > DAS source. > > > > Of course - nothing in this email constitutes me volunteering to give > such a > > tutorial .... > > > > Jim. > > > > _______________________________________________ > > DAS mailing list > > DAS at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/das > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das > From andy.jenkinson at ebi.ac.uk Mon Jan 16 13:31:52 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 16 Jan 2012 18:31:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F145407.4000900@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: I rather suspect this is a purely mental exercise, but that's fine for me ;) On 16 Jan 2012, at 16:44, Jim Procter wrote: > On 14/01/2012 01:03, Andy Jenkinson wrote: >> >> The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). > I'm not sure how different this is from the 3D security challenge from your bank/credit card company that opens in an iframe after you put your credit card details in to a secure shopping site and hit the confirm button. Authentication still has to happen in order for a transaction (in our case one involving a DAS request/response session) between the DAS server and the Ensembl server, before it renders the result in the page sent to your browser. It is similar, and the solutions we are discussing are similar. As I say, there are solutions but they require some coordination. > As for dalliance and secure servers: >> Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. > it won't actually be happening that way - take a look at this: http://www.biodalliance.org/cors.html CORS is what allows Dalliance to avoid having to communicate with DAS servers using a server in the middle (as Ensembl does). I'm saying that if a server is required to do the authentication bit (as I suspect it would be), I hope there is no technical block to having the user's computer send the signed token to the DAS server (i.e. via CORS) rather than the "Dalliance authenticator proxy". It is a slightly different model so I can't say it would definitely work. >> However, I just saw this, which is a potential solution (for OAuth at least): >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >> I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. > Interesting angle. I'm not actually sure what happens if the page from a server allowed to contact another server via CORS has been edited after it arrived in the browser sandbox - but I don't think it is really that much of an issue either way. CORS really isn't an issue - the server either allows CORS or it doesn't (in which case it is not supported by Dalliance) - it does not interact with the security side of things in any problematic way. You can use HTTP authentication in combination with CORS just fine, but we are discussing OpenID + OAuth. I am assuming that CORS is enabled on the DAS server because if it is not then Dalliance will not work. What I am talking about is the fact that Dalliance is a JS application. That is, all its code is completely editable by anyone who cares to open the Developer console, Firebug etc. There is no advantage to using OpenID+OAuth over HTTP authentication (which is what I have implemented in ProServer already so it works with Dalliance as-is) if the user can just disable the bit of the code that authenticates you. Thus we would have to ensure that the Dalliance JS only gets access to an OAuth token if an externally verifiable set of conditions is met. This is what, for me, means that there must be a server component rather than it all being done in JS. > There are two situations worth considering: > 1. an honest user wants to access secure data. In which case, they must authenticate via the dalliance host page with servers providing secure data in order to grant dalliance access via an authenticated cross-origin request from a specified server on the trusted CORS list. > 2. a black hat attempts to gain access to secure data. In which case they must fake an authentication. We then have to rely on the authentication framework detecting cracking attempts. > > The case when a (probably) honest user with valid credentials misuses the cross-origin request by messing with the javascript on the page is no different to having a user in a secure intranet who brings a virus in via a physical device or over a VPN. Since DAS is rather limited in its scope for exploits, I don't think there will be a problem with someone fiddling with the javascript, assuming, of course, the servers hosting the authenticated sources actually secured their server properly and have un-DOSable data providers at the back end... I think we're talking crossed purposes. CORS is not a way of securing servers, it's just a way of lifting the restriction placed on Javascript by browsers that it can't request URLs outside the domain of the webpage the Javascript came from. An authenticated cross-origin request (in CORS language) simply means a regular HTTP digest or HTTP basic authenticated request that happens to be setting the headers that are needed to get around this restriction. The potential for misuse is not from misusing the CORS request (indeed interfering with CORS is not possible by fiddling with the javascript). A cross-origin model is completely standard for DAS, indeed it is the whole point of having a distributed system. Nor is the misuse I am worried about to do with faking an authentication. It is simply making a trivial change in a trusted client to manipulate it into accessing secured data by bypassing the need to do the authentication step at all. OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. >>> I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. >> I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). >> > understood, and I agree, with the caveat that I'm not even sure how familiar we all are with HTTP(s) auth styles, and personally, would appreciate it if there was a practical session or tutorial on the ins and outs of authenticated DAS for those who need to do it. This could cover how different styles of authenticated DAS sources work with the existing clients (e.g. do I really have to authenticate against every secure das source every time a request is made?!). It could also involve a tutorial on setting up a secure DAS source using various schemes. For example, a tutorial on SSO systems such as Shibboleth2 as used by the UK access management federation could be useful if people needs to give specific UK academics access to a DAS source. > > Of course - nothing in this email constitutes me volunteering to give such a tutorial .... > Jim. From andy.jenkinson at ebi.ac.uk Mon Jan 16 13:34:51 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 16 Jan 2012 18:34:51 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: I agree with this. Security through obscurity should never be presented as "private". We should continue to implement some form of authentication but we should keep it simple to start with. On 16 Jan 2012, at 17:20, Bernat Gel wrote: > Hi all, > > I agree with Manuel (and I think Andy) that without a real demand for > private DAS sources implementing some complex authentication scheme might > be an overkill. However, I think that a key thing with security is that if > it's done, it has to be weel done. Somehow, a false sense of privacy (such > as the cryptic URLs in easyDAS) might be a bad thing if the user is not > aware of its weaknesses and end up unkwowingly sharing data deemed to > remain private. > > So, unless a DAS sourceis really private and secure, it should be made > clear that it is public (although maybe difficult to find). > > Bernat > > On Mon, Jan 16, 2012 at 18:10, Manuel Corpas wrote: > >> Hi guys, >> >> why not just trying to do something *really* easy that could be >> implemented in a couple of weeks rather than trying to engineer the >> perfect solution all at once? We don't really know how much demand >> there is for authenticated DAS sources, so we may end up spending a >> lot of effort for something that might be underused. >> >> The truth is that currently all DAS sources are public. Some "private >> sources" available are those provided by easyDAS with its long >> "cryptic" urls. That alone I think is already something. Any >> improvement on the ability to keep sources private, whatever small, >> would be a step in the right direction. >> >> My two cents. >> >> Manuel >> >> Manuel Corpas, PhD >> Tel: +44.122349.2372 >> Web: http://manuelcorpas.com/about/ >> Twitter: @manuelcorpas >> >> >> >> On 16 January 2012 16:44, Jim Procter >> wrote: >>> Hi Andy, and anyone else still reading :) >>> >>> >>> On 14/01/2012 01:03, Andy Jenkinson wrote: >>>> >>>> On 13 Jan 2012, at 13:04, Jim Procter wrote: >>>> >>>>> Hi. I see the dreaded secure DAS session topic as risen its head again. >>>>> >>>>> On 13/01/2012 00:42, Andy Jenkinson wrote: >>>>>> >>>>>> Originally I wanted to use a combination of OpenID and OAuth as an >>>>>> end-to-end solution. However, OpenID is based around the expectation >> that >>>>>> you are authenticating with a website using a browser - the protocol >> uses >>>>>> HTTP redirects, and OpenID providers have to have some way of telling >> you >>>>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS >> server >>>>>> that needs to check that you are who you say you are, not just the >> client. >>>>>> For a client like Ensembl, your browser simply never communicates >> with the >>>>>> DAS server so the DAS server can't get you to authenticate with the >> OpenID >>>>>> provider. >>>>> >>>>> But Ensembl *does* need to know who you are in order to request data >> that >>>>> you are allowed access to. In the OAuth model, you would have to allow >>>>> Ensembl to access privileged data from the third party DAS server, and >> that >>>>> would be achieved by the Ensembl browser presenting you with an OpenID >> login >>>>> and redirect to subsequent access control pages from the third party >> server. >>>> >>>> Not sure I follow. >>> >>> Bother! am I feeding troll(s) again ? ;) >>> >>>> The problem with the above model isn't with Ensembl it's with the DAS >>>> servers. DAS servers can't verify the user that is sitting at his >> machine >>>> (i.e. use OpenID) because they don't communicate with his machine. They >> only >>>> communicate, and can thus verify, the client (i.e. using OAuth). It >> would be >>>> better if the DAS server had some way to check the user directly (the >> client >>>> would be like a proxy for the user) but it is not possible with OpenID >> under >>>> Ensembl's current architecture (or at least it wasn't when I was >>>> investigating). So that means you have to do one of the below options >>>> instead (including having only Ensembl do the authentication). >>> >>> I'm not sure how different this is from the 3D security challenge from >> your >>> bank/credit card company that opens in an iframe after you put your >> credit >>> card details in to a secure shopping site and hit the confirm button. >>> Authentication still has to happen in order for a transaction (in our >> case >>> one involving a DAS request/response session) between the DAS server and >> the >>> Ensembl server, before it renders the result in the page sent to your >>> browser. >>> >>> As for dalliance and secure servers: >>> >>>> Basically yes, I would have thought there'd have to be some server-side >>>> Dalliance script doing most of the work, with it hopefully being >> possible to >>>> pass the signed token to the browser so that the user's machine is >> still the >>>> one issuing the DAS request. >>> >>> it won't actually be happening that way - take a look at this: >>> http://www.biodalliance.org/cors.html >>> >>>> However, I just saw this, which is a potential solution (for OAuth at >>>> least): >>>> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >>>> I'm not sure it helps however, as we need to ensure that Dalliance will >>>> only issue an authorised request to the DAS server if it has proof that >> the >>>> user has been authenticated and is on the access control list. With JS >> being >>>> editable on the client side, that check cannot be in JS. >>> >>> Interesting angle. I'm not actually sure what happens if the page from a >>> server allowed to contact another server via CORS has been edited after >> it >>> arrived in the browser sandbox - but I don't think it is really that >> much of >>> an issue either way. >>> >>> There are two situations worth considering: >>> 1. an honest user wants to access secure data. In which case, they must >>> authenticate via the dalliance host page with servers providing secure >> data >>> in order to grant dalliance access via an authenticated cross-origin >> request >>> from a specified server on the trusted CORS list. >>> 2. a black hat attempts to gain access to secure data. In which case they >>> must fake an authentication. We then have to rely on the authentication >>> framework detecting cracking attempts. >>> >>> The case when a (probably) honest user with valid credentials misuses the >>> cross-origin request by messing with the javascript on the page is no >>> different to having a user in a secure intranet who brings a virus in >> via a >>> physical device or over a VPN. Since DAS is rather limited in its scope >> for >>> exploits, I don't think there will be a problem with someone fiddling >> with >>> the javascript, assuming, of course, the servers hosting the >> authenticated >>> sources actually secured their server properly and have un-DOSable data >>> providers at the back end... >>> >>>>>> Lastly, neither solution works for daemon-style clients (e.g. >>>>>> command-line analysis applications where the user is not present), >> again >>>>>> because they can't use OpenID. The catch-all solution is to use X.509 >>>>>> certificates (public/private key cryptography) but it is heavyweight >> and >>>>>> probably too complicated to provide a good user experience. Truth be >> told, >>>>>> it has proved difficult to discuss this topic amongst the community >> because >>>>>> it gets technically very complex. >>>>> >>>>> I'd be very much in favour of people who *need* to achieve this >> spending >>>>> some time during the developer days with an invited expert (with >> special >>>>> anti-trolling skills to counter folk like me) in a closed session, in >> order >>>>> to identify components that would enable both browser and non-browser >> based >>>>> clients to work with OAuth, and set up a trial OAuth DAS source >> network for >>>>> testing. There are libraries that support OAuth ( >> http://oauth.net/code/) >>>>> both for providers and consumers, but DAS client libraries will need >>>>> extension to allow secure negotiation and signed DAS HTTP interaction, >> and >>>>> their APIs will need additional parameters/methods to allow session >>>>> management. >>>> >>>> I have no objections to participating in principle, but I remain to >>>> convinced it is worth the effort (i.e. that HTTP auth is insufficient >> and >>>> that OAuth would be adopted). >>>> >>> understood, and I agree, with the caveat that I'm not even sure how >> familiar >>> we all are with HTTP(s) auth styles, and personally, would appreciate it >> if >>> there was a practical session or tutorial on the ins and outs of >>> authenticated DAS for those who need to do it. This could cover how >>> different styles of authenticated DAS sources work with the existing >> clients >>> (e.g. do I really have to authenticate against every secure das source >> every >>> time a request is made?!). It could also involve a tutorial on setting >> up a >>> secure DAS source using various schemes. For example, a tutorial on SSO >>> systems such as Shibboleth2 as used by the UK access management >> federation >>> could be useful if people needs to give specific UK academics access to a >>> DAS source. >>> >>> Of course - nothing in this email constitutes me volunteering to give >> such a >>> tutorial .... >>> >>> Jim. >>> >>> _______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das >> >> _______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das >> > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Mon Jan 16 16:12:52 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Mon, 16 Jan 2012 21:12:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: On 16 January 2012 18:31, Andy Jenkinson wrote: > I rather suspect this is a purely mental exercise, but that's fine for me ;) > OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. Right, the person who 'owns' the data (i.e. a list of contacts hosted on a Google account) explicitly grants the third party 'app' permission to access the data (via the account) in a specified way (as defined by the Google APIs). That app can then email all your contacts in the middle of the night while you're sleeping, but you trust that that won't happen when you click the 'grant' button. i.e. I (the verified me) can grant Ensembl permission to access my SNP genotype data from 23andMe (hah), and I trust Ensemble not to do anything nasty with that data when I log off. Although it's a bit of a pain to set this up, the point is that literally thousands of app developers (including me) have done it before, and there are hundreds of docs. Here is where I started when I built a command line twitter bot: https://dev.twitter.com/docs/auth I'm not trying to say its quick and easy to do and everyone should do it like this, I just thought I'd provide the above encapsulation, which hopefully isn't too far from how it could be done. Cheers, Dan. From dan.bolser at gmail.com Mon Jan 16 18:16:57 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Mon, 16 Jan 2012 23:16:57 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Heh... it doesn't work: https://github.com/dbolser/TwitterBot---nowlistening-Perl-script-for-xmms Not sure what I'm doing wrong. On 16 January 2012 21:12, Dan Bolser wrote: > On 16 January 2012 18:31, Andy Jenkinson wrote: >> I rather suspect this is a purely mental exercise, but that's fine for me ;) > > > >> OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. > > Right, the person who 'owns' the data (i.e. a list of contacts hosted > on a Google account) explicitly grants the third party 'app' > permission to access the data (via the account) in a specified way (as > defined by the Google APIs). That app can then email all your contacts > in the middle of the night while you're sleeping, but you trust that > that won't happen when you click the 'grant' button. > > i.e. I (the verified me) can grant Ensembl permission to access my SNP > genotype data from 23andMe (hah), and I trust Ensemble not to do > anything nasty with that data when I log off. > > Although it's a bit of a pain to set this up, the point is that > literally thousands of app developers (including me) have done it > before, and there are hundreds of docs. Here is where I started when I > built a command line twitter bot: > https://dev.twitter.com/docs/auth > > > I'm not trying to say its quick and easy to do and everyone should do > it like this, I just thought I'd provide the above encapsulation, > which hopefully isn't too far from how it could be done. > > > Cheers, > Dan. From jw12 at sanger.ac.uk Tue Jan 17 05:44:40 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Tue, 17 Jan 2012 10:44:40 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: <5867A14E-E3D8-4E9A-94F5-83AFEA60FF2F@sanger.ac.uk> Hi I'd be against using OAuth because it will probably suffer from the same limitations that OpenId suffered from. 1) It will be the in thing for a while with support for certain programming languages or environments (such as browsers) but will soon go out of fashion when something else with a cool sounding name comes along :) 2) It takes us away even more than we already are from the principle that the DAS system should be simple. 3) If it's complicated it can be difficult to change your application if the environment that you are running your application in changes or other factors make you change it (e.g. when the sanger web site no longer supported sticky sessions I had to try and rewrite the openID code for the registry which turned out to be dependent on normal in memory Java sessions, so I had to change code that I didn't really understand, thus introducing security holes, so I ended up turning to good old username and passwords). 4) Everybody understands username and passwords boxes in an user interface whereas if we start going on about OAuth authentication and using their facebook accounts people will get frightened off - I'm also not sure I won't my gmail or facebook accounts to log me in to the DAS registry automatically and start distributing my contact lists ;) My preference and I think most other peoples when we have been on this discussion before (3rd time around??) is to use basic https security that has been around for years. I also prefer the option of sending the username and passwords for every request for a secure data source as the server then remains in one state (the registry sends them in the headers, MyDAS currently sends them in the xml - we need to decide which to use). This would be in preference to an amazon S3 type system where tokens are given for a limited period of time to the client/user (the S3 type system would be my second option). These would be a good balance between ease of development/understanding and security. If specific clients and servers wanted to add extra security they could add IP restrictions so they know the client is hosted at a specific place. Cheers Jonathan. On 16 Jan 2012, at 23:16, Dan Bolser wrote: > Heh... it doesn't work: > > https://github.com/dbolser/TwitterBot---nowlistening-Perl-script-for-xmms > > Not sure what I'm doing wrong. > > > On 16 January 2012 21:12, Dan Bolser wrote: >> On 16 January 2012 18:31, Andy Jenkinson >> wrote: >>> I rather suspect this is a purely mental exercise, but that's fine >>> for me ;) >> >> >> >>> OAuth is entirely based upon the notion that the server with the >>> data (e.g. Google Contacts) can trust the application (e.g. the >>> Android Contacts app) to do the right thing with the data. There >>> is no requirement that the person who owns the data, or any other >>> person, has to be present, and the application doesn't have to >>> prove that this will happen. It just has to get the user to agree >>> that the application can be trusted. It's up to us therefore to >>> provide a secure link between OpenID and OAuth. >> >> Right, the person who 'owns' the data (i.e. a list of contacts hosted >> on a Google account) explicitly grants the third party 'app' >> permission to access the data (via the account) in a specified way >> (as >> defined by the Google APIs). That app can then email all your >> contacts >> in the middle of the night while you're sleeping, but you trust that >> that won't happen when you click the 'grant' button. >> >> i.e. I (the verified me) can grant Ensembl permission to access my >> SNP >> genotype data from 23andMe (hah), and I trust Ensemble not to do >> anything nasty with that data when I log off. >> >> Although it's a bit of a pain to set this up, the point is that >> literally thousands of app developers (including me) have done it >> before, and there are hundreds of docs. Here is where I started >> when I >> built a command line twitter bot: >> https://dev.twitter.com/docs/auth >> >> >> I'm not trying to say its quick and easy to do and everyone should do >> it like this, I just thought I'd provide the above encapsulation, >> which hopefully isn't too far from how it could be done. >> >> >> Cheers, >> Dan. > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Fri Jan 20 05:41:18 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Fri, 20 Jan 2012 10:41:18 +0000 Subject: [DAS] DAS Workshop Registrations Message-ID: <0F5CE38F-8FC4-4AEC-9BC4-6EB76BCA4850@sanger.ac.uk> Hi If you are intending to come to the DAS workshop this year can you register soon as we would like to get an idea of numbers for the 3 respective days (1st day tutorials and short talks, 2nd and 3rd day developers hackathon). This will then enable us to start firming up the schedule for the 3 days. If you are not intending to come to the workshop can I ask for some feedback that may help us entice you in the future? Many thanks Jonathan. See workshop email for you convenience below: DAS is currently being used to share annotations on genomes, protein alignments, structural and interaction information. If you are interested in sharing biological information the DAS workshop below may be of interest to you. Registration is open for the 2012 DAS workshop (27-29 February) at the Genome Campus, Hinxton UK. If you are interested in attending, please find out more by going to http://www.ebi.ac.uk/training/onsite/120227_DAS.html and register via the web link at the bottom of the page. This workshop will cater for novice to expert DAS users as each day is optional. Please register early as places will be limited. Registration closes 10 February 2012 - 12:00. If you are interested in giving a 15 minute talk on the second day please email Jonathan Warren using jonathan.warren at sanger.ac.uk Many thanks The Sanger/EBI DAS team. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Thu Jan 5 09:57:05 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 5 Jan 2012 09:57:05 +0000 Subject: [DAS] workshop email Message-ID: Hi DAS people. I intend to circulate to other email lists the DAS workshop email just sent. If anyone has suggestions as to other email lists to post it to or would like to forward this email on to them (saves me registering on lots of lists :) ) then please do (but let me know so I don't duplicate posts). List I intend to use are: gmod biojava ensembl bioperl Sanger/EBI campus We could do with it being circulated around less developer focused lists e.g. plant focused user groups? More general biology lists? Suggestions? Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Thu Jan 5 09:49:02 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 5 Jan 2012 09:49:02 +0000 Subject: [DAS] Registrations for DAS Workshop 2012 Message-ID: <69FA2B25-F15E-4361-BBAC-E6DC3975F435@sanger.ac.uk> DAS is currently being used to share annotations on genomes, protein alignments, structural and interaction information. If you are interested in sharing biological information the DAS workshop below may be of interest to you. Learn of and contribute to current developments in DAS such as: DAS in the cloud, DAS for Genotype Data, DAS searching, DAS for collaborative annotation projects, DAS alternative formats. Registration is open for the 2012 DAS workshop (27-29 February) at the Genome Campus, Hinxton UK. If you are interested in attending, please find out more by going to http://www.ebi.ac.uk/training/onsite/120227_DAS.html and register via the web link at the bottom of the page. This workshop will cater for novice to expert DAS users as each day is optional. Please register early as places will be limited. Registration closes 10 February 2012 - 12:00. If you are interested in giving a 15 minute talk on the second day please email Jonathan Warren using jonathan.warren at sanger.ac.uk Many thanks The Sanger/EBI DAS team. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Wed Jan 11 14:20:54 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Wed, 11 Jan 2012 14:20:54 +0000 Subject: [DAS] Workshop 2012 format change Message-ID: <18968EB6-4CE3-489E-88E6-31D206AFFC7C@sanger.ac.uk> After some discussions and partly due to a lack of volunteers for talks we have decided to modify the DAS workshop format this year. The first day as per previous years will be tutorials and talks for new users of DAS. The second and third day will be for DAS developers as a hackathon to facilitate the implementation and updating of clients, servers and libraries to implement the latest 1.6E specifications. Thus for this year we are dispensing with the second day of talks which will hopefully be back in some form next year. The workshop registration page and registrations will be updated to reflect this change soon. Any comments or suggestions please let us know. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Thu Jan 12 13:21:14 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 12 Jan 2012 13:21:14 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps Message-ID: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Hi I've put some instructions for people with little or no technical ability and no access to IT personnel or servers, to be able to publish there genotype data from companies such as 23andme etc as a DAS source on the amazon cloud. http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ All someone needs is a tab delimited text file from one of these companies, a credit card and an internet connection. The instructions show them how to set up and deploy their server to the cloud in 6 easy steps and then view the data in Ensembl. Example server can be accessed from here http://mychoiceofname.elasticbeanstalk.com/das/person1 Any comments suggestions welcomed. Cheers Jonathan. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From dan.bolser at gmail.com Thu Jan 12 13:43:59 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 13:43:59 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Message-ID: That's great! Just last night I was looking for something that would fulfill this requirement. One suggestion would be for authentication like I get with my Google, Flicker, Twitter, Facebook, etc. accounts. (Is this an open protocol for authentication, or did they each re-invent this wheel?) i.e. I want to be able to specifically grant access to my data by a known third party. e.g. GeneticGuru wants access to your GenotypeArchive account, allow or deny? Do any existing personal genotype archives have this kind of 'genotype server' service, or is DAS the first system to provide this functionality? Do you plan to build it into a dedicated service somewhere? Cheers, Dan. On 12 January 2012 13:21, Jonathan Warren wrote: > Hi > > I've put some instructions for people with little or no technical ability > and no access to IT personnel or servers, to be able to publish there > genotype data from companies such as 23andme etc as a DAS source on the > amazon cloud. > > http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ > > All someone needs is a tab delimited text file from one of these companies, > a credit card and an internet connection. The instructions show them how to > set up and deploy their server to the cloud in 6 easy steps and then view > the data in Ensembl. > > Example server can be accessed from here > http://mychoiceofname.elasticbeanstalk.com/das/person1 > > Any comments suggestions welcomed. > > Cheers > > Jonathan. > > Jonathan Warren > Senior Developer and DAS coordinator > blog: http://biodasman.wordpress.com/ > jw12 at sanger.ac.uk > Ext: 2314 > Telephone: 01223 492314 > > > > > > > > > > > -- > The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a > charity registered in England with number 1021457 and acompany registered in > England with number 2742969, whose registeredoffice is 215 Euston Road, > London, NW1 2BE._______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From jw12 at sanger.ac.uk Thu Jan 12 15:31:52 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Thu, 12 Jan 2012 15:31:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> Message-ID: <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> On 12 Jan 2012, at 13:43, Dan Bolser wrote: Hi Dan Thanks for the feedback. > That's great! > > Just last night I was looking for something that would fulfill this > requirement. > > One suggestion would be for authentication like I get with my Google, > Flicker, Twitter, Facebook, etc. accounts. (Is this an open protocol > for authentication, or did they each re-invent this wheel?) > > i.e. I want to be able to specifically grant access to my data by a > known third party. > > e.g. GeneticGuru wants access to your GenotypeArchive account, allow > or deny? > For the situation where someone only wants themselves to access there own data for the moment they can just turn on their DAS server when they want to use it and not advertise their url. But if there is demand for people to be able to specify who can use it I can add some security to this. We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. The registry already supports this for the web service and I believe Andy J has implemented it in proserver as well. MyDAS if it doesn't already have it, can be easily modified to do so I think and it's on my list to do this for a writeback instance I'm setting up for genomic data anyway. However the big fly in the ointment is that I'm pretty sure no clients support it at the moment however. I'm sure ensembl doesn't and I don't think Dalliance does, GBrowse doesn't? I think maybe Web Apollo does, but I haven't heard anything from those guys in ages? I hope I'm wrong on this though. Maybe Karyodas can implement it quickly???? http://mykaryoview.com > Do any existing personal genotype archives have this kind of 'genotype > server' service, or is DAS the first system to provide this > functionality? Do you plan to build it into a dedicated service > somewhere? I'm in discussions about this with the powers that be at the moment. Problems would include massive memory usage if implemented this way and also legal issues around managing other peoples genomic data. Which is why the personal virtual machine in the cloud is something that is quite an elegant solution at the moment :) I would like the process for people to be even easier though and for setting up DAS sources in general. > > > Cheers, > Dan. > > On 12 January 2012 13:21, Jonathan Warren wrote: >> Hi >> >> I've put some instructions for people with little or no technical >> ability >> and no access to IT personnel or servers, to be able to publish there >> genotype data from companies such as 23andme etc as a DAS source on >> the >> amazon cloud. >> >> http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ >> >> All someone needs is a tab delimited text file from one of these >> companies, >> a credit card and an internet connection. The instructions show >> them how to >> set up and deploy their server to the cloud in 6 easy steps and >> then view >> the data in Ensembl. >> >> Example server can be accessed from here >> http://mychoiceofname.elasticbeanstalk.com/das/person1 >> >> Any comments suggestions welcomed. >> >> Cheers >> >> Jonathan. >> >> Jonathan Warren >> Senior Developer and DAS coordinator >> blog: http://biodasman.wordpress.com/ >> jw12 at sanger.ac.uk >> Ext: 2314 >> Telephone: 01223 492314 >> >> >> >> >> >> >> >> >> >> >> -- >> The Wellcome Trust Sanger Institute is operated by Genome >> ResearchLimited, a >> charity registered in England with number 1021457 and acompany >> registered in >> England with number 2742969, whose registeredoffice is 215 Euston >> Road, >> London, NW1 2BE._______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From andy.jenkinson at ebi.ac.uk Thu Jan 12 17:07:50 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Thu, 12 Jan 2012 17:07:50 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> Message-ID: <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Great stuff Jon! On 12 Jan 2012, at 15:31, Jonathan Warren wrote: > > On 12 Jan 2012, at 13:43, Dan Bolser wrote: > >> I want to be able to specifically grant access to my data by a known third party. > > We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. Standard HTTP authentication by contrast is much easier to implement and is a sort of "easy win" for the interim. > The registry already supports this for the web service and I believe Andy J has implemented it in proserver as well. MyDAS if it doesn't already have it, can be easily modified to do so I think and it's on my list to do this for a writeback instance I'm setting up for genomic data anyway. > > However the big fly in the ointment is that I'm pretty sure no clients support it at the moment however. I'm sure ensembl doesn't and I don't think Dalliance does, GBrowse doesn't? I think maybe Web Apollo does, but I haven't heard anything from those guys in ages? I hope I'm wrong on this though. Maybe Karyodas can implement it quickly???? http://mykaryoview.com Actually Dalliance does support HTTP authentication because, being pure Javascript, it uses the browser's implementation. The development branch of ProServer includes support for HTTP authentication and SSL encryption. I'm fairly sure the authentication (basic and digest) work well, but I think the encryption could use some testing. The main barrier though is as Jonathan says that no other clients support it, it would be easier to debug using these rather than using browsers directly. Obviously what we really want is for Ensembl to implement it. I'm not sure what it would take for myKaryoview and Dasty, they are javascript clients but I think they use a server-side proxy so it might be hard/impossible for them to use the browser's cache of passwords. That means storing them locally, which means some sort of login system, which means extra security considerations... etc. I think Jonathan's personal genomics server is a clear use case for authentication, and I would also seek to incorporate it into EasyDAS (Which uses ProServer). We need major clients to adopt it, and it is of course the classic DAS chicken and egg situation. Cheers, Andy > >> Do any existing personal genotype archives have this kind of 'genotype >> server' service, or is DAS the first system to provide this >> functionality? Do you plan to build it into a dedicated service >> somewhere? > I'm in discussions about this with the powers that be at the moment. Problems would include massive memory usage if implemented this way and also legal issues around managing other peoples genomic data. Which is why the personal virtual machine in the cloud is something that is quite an elegant solution at the moment :) I would like the process for people to be even easier though and for setting up DAS sources in general. > >> >> >> Cheers, >> Dan. >> >> On 12 January 2012 13:21, Jonathan Warren wrote: >>> Hi >>> >>> I've put some instructions for people with little or no technical ability >>> and no access to IT personnel or servers, to be able to publish there >>> genotype data from companies such as 23andme etc as a DAS source on the >>> amazon cloud. >>> >>> http://biodasman.wordpress.com/2012/01/12/easy-deployment-of-das-server-for-personal-genotype-data-to-the-amazon-cloud/ >>> >>> All someone needs is a tab delimited text file from one of these companies, >>> a credit card and an internet connection. The instructions show them how to >>> set up and deploy their server to the cloud in 6 easy steps and then view >>> the data in Ensembl. >>> >>> Example server can be accessed from here >>> http://mychoiceofname.elasticbeanstalk.com/das/person1 >>> >>> Any comments suggestions welcomed. >>> >>> Cheers >>> >>> Jonathan. >>> >>> Jonathan Warren >>> Senior Developer and DAS coordinator >>> blog: http://biodasman.wordpress.com/ >>> jw12 at sanger.ac.uk >>> Ext: 2314 >>> Telephone: 01223 492314 >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a >>> charity registered in England with number 1021457 and acompany registered in >>> England with number 2742969, whose registeredoffice is 215 Euston Road, >>> London, NW1 2BE._______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das > > Jonathan Warren > Senior Developer and DAS coordinator > blog: http://biodasman.wordpress.com/ > jw12 at sanger.ac.uk > Ext: 2314 > Telephone: 01223 492314 > > > > > > > > > > > -- > The Wellcome Trust Sanger Institute is operated by Genome ResearchLimited, a charity registered in England with number 1021457 and acompany registered in England with number 2742969, whose registeredoffice is 215 Euston Road, London, NW1 2BE._______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Thu Jan 12 18:08:01 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 18:08:01 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Message-ID: On 12 January 2012 17:07, Andy Jenkinson wrote: > Great stuff Jon! > > On 12 Jan 2012, at 15:31, Jonathan Warren wrote: >> >> On 12 Jan 2012, at 13:43, Dan Bolser wrote: >> >>> I want to be able to specifically grant access to my data by a known third party. >> >> We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. > > I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. AFAIK, using something like the above, you authenticate with the client using OpenID, and the client is authenticated to access your data via OAuth. You can then build your client to allow various levels of sharing with other users in the system, as with FB. Would building OAuth into Proserver, then identifying with OpenID be a way round the 'technical reasons' you described above? Or is it just running in circles? Cheers, Dan. From dan.bolser at gmail.com Thu Jan 12 18:09:48 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Thu, 12 Jan 2012 18:09:48 +0000 Subject: [DAS] DAS2GFF3? Message-ID: Hi, Given a set of features in DAS format, what is the easiest way to generate GFF3 format? Can most DAS server implementation be persuaded to serve GFF3 in response to DAS feature requests? Cheers, Dan. From andy.jenkinson at ebi.ac.uk Fri Jan 13 00:42:50 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Fri, 13 Jan 2012 00:42:50 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> Message-ID: <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> On 12 Jan 2012, at 18:08, Dan Bolser wrote: > On 12 January 2012 17:07, Andy Jenkinson wrote: >> Great stuff Jon! >> >> On 12 Jan 2012, at 15:31, Jonathan Warren wrote: >>> >>> On 12 Jan 2012, at 13:43, Dan Bolser wrote: >>> >>>> I want to be able to specifically grant access to my data by a known third party. >>> >>> We had large debates about how to implement security in DAS at the last couple of DAS workshops. In the end it was decided we would go with BASIC authentication and https requests and responses and people would have to trust DAS clients with their username and passwords. >> >> I believe those providers use (or are migrating to) a common authorisation protocol based on OAuth. This type of authorisation actually only allows you to control which -applications- have access to your data, not which individuals. That means each individual client needs to be configured for this purpose. Really what is needed is an end-to-end solution across both clients and servers, with a common authentication/identification mechanism and across multiple providers. Particularly the authentication part is difficult because, for technical reasons, we can't use OpenID. It'd be great and there are potential solutions, but the "activation energy" and coordination required is quite high. > > AFAIK, using something like the above, you authenticate with the > client using OpenID, and the client is authenticated to access your > data via OAuth. You can then build your client to allow various levels > of sharing with other users in the system, as with FB. > > Would building OAuth into Proserver, then identifying with OpenID be a > way round the 'technical reasons' you described above? Or is it just > running in circles? Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still setting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had the resources to do it. Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. Cheers, Andy From jprocter at compbio.dundee.ac.uk Fri Jan 13 13:04:43 2012 From: jprocter at compbio.dundee.ac.uk (Jim Procter) Date: Fri, 13 Jan 2012 13:04:43 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> Message-ID: <4F102BEB.1020101@compbio.dundee.ac.uk> Hi. I see the dreaded secure DAS session topic as risen its head again. On 13/01/2012 00:42, Andy Jenkinson wrote: > Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. > It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still set! > ting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. Yes. security isn't cheap it seems :) > I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had ! > the resources to do it. So the problem here with OAuth is that pure browser clients need a secure store for authenticating a user's access to a particular resource ? I don't think there is any way around that either, and I think the onus to provide this is the hosting page of the client - i.e. dalliance.org would need to be recognised as a peer on a secure DAS OAuth network (using the servers own keys). Then, users wanting access to secure data would log in to the DAS source independently via a secure session on dalliance.org, which would then allow dalliance to browse the data from the secure server. If someone wants to set up a Dalliance browser in their own OAuth trust network, then they would need to have their own Dalliance hosting server and make it known to the other peers accordingly. > Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. Jim. From andy.jenkinson at ebi.ac.uk Sat Jan 14 01:03:40 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Sat, 14 Jan 2012 01:03:40 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F102BEB.1020101@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> Message-ID: <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> On 13 Jan 2012, at 13:04, Jim Procter wrote: > Hi. I see the dreaded secure DAS session topic as risen its head again. > > On 13/01/2012 00:42, Andy Jenkinson wrote: >> Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. > But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. Not sure I follow. The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). >> It would be possible instead to have a system whereby you configure the DAS server to authorise a piece of software via OAuth, and have the client take care of making sure only certain users can access that data source. You are putting the onus of deciding which applications to trust onto the owner of the data, which is not ideal for data like personal genomics (although certainly not worse than handing out passwords). Obviously this has major implications for every client that wants to support it, because it's a whole extra suite of functionality they all need to implement. There would need to be a way for the clients to know who gets to decide who can access the source, though it could be implemented via some DAS-specific way easily I imagine. I think if we were to do it, possibly the best way is actually to have the assignment of who can access the data determined at the DAS server (a list of OpenID identities) which clients can simply query for. That way you're still s! > et! >> ting the permissions on the client. You're still trusting the client to honour the list (which includes not caching it), but at least the clients don't need to maintain the access control list. However they still need to implement OpenID. That's a technical requirement but, in the case of Ensembl which already providers user login facilities, it has other consequences. Convincing people it's worth adopting DAS is one thing, but convincing them that they must also use OpenID for their login system is another. All this is why I say it's possible, but there is a significant investment to get it all up and running. > Yes. security isn't cheap it seems :) >> I'm also not sure if pure browser-implemented clients like Dalliance can use this method, both OpenID and OAuth involve signing messages with the application's secret key, and it's difficult to do that (and store these keys) without a server of some kind. They'd probably be forced into using one. Still, this is my preferred solution if everybody was on board and had ! >> the resources to do it. > So the problem here with OAuth is that pure browser clients need a secure store for authenticating a user's access to a particular resource ? I don't think there is any way around that either, and I think the onus to provide this is the hosting page of the client - i.e. dalliance.org would need to be recognised as a peer on a secure DAS OAuth network (using the servers own keys). Then, users wanting access to secure data would log in to the DAS source independently via a secure session on dalliance.org, which would then allow dalliance to browse the data from the secure server. If someone wants to set up a Dalliance browser in their own OAuth trust network, then they would need to have their own Dalliance hosting server and make it known to the other peers accordingly. Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. However, I just saw this, which is a potential solution (for OAuth at least): http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. >> Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. > I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). > Jim. > > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From jprocter at compbio.dundee.ac.uk Mon Jan 16 16:44:55 2012 From: jprocter at compbio.dundee.ac.uk (Jim Procter) Date: Mon, 16 Jan 2012 16:44:55 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> Message-ID: <4F145407.4000900@compbio.dundee.ac.uk> Hi Andy, and anyone else still reading :) On 14/01/2012 01:03, Andy Jenkinson wrote: > On 13 Jan 2012, at 13:04, Jim Procter wrote: > >> Hi. I see the dreaded secure DAS session topic as risen its head again. >> >> On 13/01/2012 00:42, Andy Jenkinson wrote: >>> Originally I wanted to use a combination of OpenID and OAuth as an end-to-end solution. However, OpenID is based around the expectation that you are authenticating with a website using a browser - the protocol uses HTTP redirects, and OpenID providers have to have some way of telling you are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server that needs to check that you are who you say you are, not just the client. For a client like Ensembl, your browser simply never communicates with the DAS server so the DAS server can't get you to authenticate with the OpenID provider. >> But Ensembl *does* need to know who you are in order to request data that you are allowed access to. In the OAuth model, you would have to allow Ensembl to access privileged data from the third party DAS server, and that would be achieved by the Ensembl browser presenting you with an OpenID login and redirect to subsequent access control pages from the third party server. > Not sure I follow. Bother! am I feeding troll(s) again ? ;) > The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). I'm not sure how different this is from the 3D security challenge from your bank/credit card company that opens in an iframe after you put your credit card details in to a secure shopping site and hit the confirm button. Authentication still has to happen in order for a transaction (in our case one involving a DAS request/response session) between the DAS server and the Ensembl server, before it renders the result in the page sent to your browser. As for dalliance and secure servers: > Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. it won't actually be happening that way - take a look at this: http://www.biodalliance.org/cors.html > However, I just saw this, which is a potential solution (for OAuth at least): > http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ > I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. Interesting angle. I'm not actually sure what happens if the page from a server allowed to contact another server via CORS has been edited after it arrived in the browser sandbox - but I don't think it is really that much of an issue either way. There are two situations worth considering: 1. an honest user wants to access secure data. In which case, they must authenticate via the dalliance host page with servers providing secure data in order to grant dalliance access via an authenticated cross-origin request from a specified server on the trusted CORS list. 2. a black hat attempts to gain access to secure data. In which case they must fake an authentication. We then have to rely on the authentication framework detecting cracking attempts. The case when a (probably) honest user with valid credentials misuses the cross-origin request by messing with the javascript on the page is no different to having a user in a secure intranet who brings a virus in via a physical device or over a VPN. Since DAS is rather limited in its scope for exploits, I don't think there will be a problem with someone fiddling with the javascript, assuming, of course, the servers hosting the authenticated sources actually secured their server properly and have un-DOSable data providers at the back end... >>> Lastly, neither solution works for daemon-style clients (e.g. command-line analysis applications where the user is not present), again because they can't use OpenID. The catch-all solution is to use X.509 certificates (public/private key cryptography) but it is heavyweight and probably too complicated to provide a good user experience. Truth be told, it has proved difficult to discuss this topic amongst the community because it gets technically very complex. >> I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. > I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). > understood, and I agree, with the caveat that I'm not even sure how familiar we all are with HTTP(s) auth styles, and personally, would appreciate it if there was a practical session or tutorial on the ins and outs of authenticated DAS for those who need to do it. This could cover how different styles of authenticated DAS sources work with the existing clients (e.g. do I really have to authenticate against every secure das source every time a request is made?!). It could also involve a tutorial on setting up a secure DAS source using various schemes. For example, a tutorial on SSO systems such as Shibboleth2 as used by the UK access management federation could be useful if people needs to give specific UK academics access to a DAS source. Of course - nothing in this email constitutes me volunteering to give such a tutorial .... Jim. From mc at manuelcorpas.com Mon Jan 16 17:10:29 2012 From: mc at manuelcorpas.com (Manuel Corpas) Date: Mon, 16 Jan 2012 17:10:29 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F145407.4000900@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Hi guys, why not just trying to do something *really* easy that could be implemented in a couple of weeks rather than trying to engineer the perfect solution all at once? We don't really know how much demand there is for authenticated DAS sources, so we may end up spending a lot of effort for something that might be underused. The truth is that currently all DAS sources are public. Some "private sources" available are those provided by easyDAS with its long "cryptic" urls. That alone I think is already something. Any improvement on the ability to keep sources private, whatever small, would be a step in the right direction. My two cents. Manuel Manuel Corpas, PhD Tel:? ? ? +44.122349.2372 Web: ? ?http://manuelcorpas.com/about/ Twitter: @manuelcorpas On 16 January 2012 16:44, Jim Procter wrote: > Hi Andy, and anyone else still reading :) > > > On 14/01/2012 01:03, Andy Jenkinson wrote: >> >> On 13 Jan 2012, at 13:04, Jim Procter wrote: >> >>> Hi. I see the dreaded secure DAS session topic as risen its head again. >>> >>> On 13/01/2012 00:42, Andy Jenkinson wrote: >>>> >>>> Originally I wanted to use a combination of OpenID and OAuth as an >>>> end-to-end solution. However, OpenID is based around the expectation that >>>> you are authenticating with a website using a browser - the protocol uses >>>> HTTP redirects, and OpenID providers have to have some way of telling you >>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS server >>>> that needs to check that you are who you say you are, not just the client. >>>> For a client like Ensembl, your browser simply never communicates with the >>>> DAS server so the DAS server can't get you to authenticate with the OpenID >>>> provider. >>> >>> But Ensembl *does* need to know who you are in order to request data that >>> you are allowed access to. In the OAuth model, you would have to allow >>> Ensembl to access privileged data from the third party DAS server, and that >>> would be achieved by the Ensembl browser presenting you with an OpenID login >>> and redirect to subsequent access control pages from the third party server. >> >> Not sure I follow. > > Bother! am I feeding troll(s) again ? ;) > >> The problem with the above model isn't with Ensembl it's with the DAS >> servers. DAS servers can't verify the user that is sitting at his machine >> (i.e. use OpenID) because they don't communicate with his machine. They only >> communicate, and can thus verify, the client (i.e. using OAuth). It would be >> better if the DAS server had some way to check the user directly (the client >> would be like a proxy for the user) but it is not possible with OpenID under >> Ensembl's current architecture (or at least it wasn't when I was >> investigating). So that means you have to do one of the below options >> instead (including having only Ensembl do the authentication). > > I'm not sure how different this is from the 3D security challenge from your > bank/credit card company that opens in an iframe after you put your credit > card details in to a secure shopping site and hit the confirm button. > Authentication still has to happen in order for a transaction (in our case > one involving a DAS request/response session) between the DAS server and the > Ensembl server, before it renders the result in the page sent to your > browser. > > As for dalliance and secure servers: > >> Basically yes, I would have thought there'd have to be some server-side >> Dalliance script doing most of the work, with it hopefully being possible to >> pass the signed token to the browser so that the user's machine is still the >> one issuing the DAS request. > > it won't actually be happening that way - take a look at this: > http://www.biodalliance.org/cors.html > >> However, I just saw this, which is a potential solution (for OAuth at >> least): >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >> I'm not sure it helps however, as we need to ensure that Dalliance will >> only issue an authorised request to the DAS server if it has proof that the >> user has been authenticated and is on the access control list. With JS being >> editable on the client side, that check cannot be in JS. > > Interesting angle. I'm not actually sure what happens if the page from a > server allowed to contact another server via CORS has been edited after it > arrived in the browser sandbox - but I don't think it is really that much of > an issue either way. > > There are two situations worth considering: > 1. an honest user wants to access secure data. In which case, they must > authenticate via the dalliance host page with servers providing secure data > in order to grant dalliance access via an authenticated cross-origin request > from a specified server on the trusted CORS list. > 2. a black hat attempts to gain access to secure data. In which case they > must fake an authentication. We then have to rely on the authentication > framework detecting cracking attempts. > > The case when a (probably) honest user with valid credentials misuses the > cross-origin request by messing with the javascript on the page is no > different to having a user in a secure intranet who brings a virus in via a > physical device or over a VPN. Since DAS is rather limited in its scope for > exploits, I don't think there will be a problem with someone fiddling with > the javascript, assuming, of course, the servers hosting the authenticated > sources actually secured their server properly and have un-DOSable data > providers at the back end... > >>>> Lastly, neither solution works for daemon-style clients (e.g. >>>> command-line analysis applications where the user is not present), again >>>> because they can't use OpenID. The catch-all solution is to use X.509 >>>> certificates (public/private key cryptography) but it is heavyweight and >>>> probably too complicated to provide a good user experience. Truth be told, >>>> it has proved difficult to discuss this topic amongst the community because >>>> it gets technically very complex. >>> >>> I'd be very much in favour of people who *need* to achieve this spending >>> some time during the developer days with an invited expert (with special >>> anti-trolling skills to counter folk like me) in a closed session, in order >>> to identify components that would enable both browser and non-browser based >>> clients to work with OAuth, and set up a trial OAuth DAS source network for >>> testing. There are libraries that support OAuth (http://oauth.net/code/) >>> both for providers and consumers, but DAS client libraries will need >>> extension to allow secure negotiation and signed DAS HTTP interaction, and >>> their APIs will need additional parameters/methods to allow session >>> management. >> >> I have no objections to participating in principle, but I remain to >> convinced it is worth the effort (i.e. that HTTP auth is insufficient and >> that OAuth would be adopted). >> > understood, and I agree, with the caveat that I'm not even sure how familiar > we all are with HTTP(s) auth styles, and personally, would appreciate it if > there was a practical session or tutorial on the ins and outs of > authenticated DAS for those who need to do it. This could cover how > different styles of authenticated DAS sources work with the existing clients > (e.g. do I really have to authenticate against every secure das source every > time a request is made?!). It could also involve a tutorial on setting up a > secure DAS source using various schemes. For example, a tutorial on SSO > systems such as Shibboleth2 as used by the UK access management federation > could be useful if people needs to give specific UK academics access to a > DAS source. > > Of course - nothing in this email constitutes me volunteering to give such a > tutorial .... > > Jim. > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From bernatgel at gmail.com Mon Jan 16 17:20:55 2012 From: bernatgel at gmail.com (Bernat Gel) Date: Mon, 16 Jan 2012 18:20:55 +0100 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Hi all, I agree with Manuel (and I think Andy) that without a real demand for private DAS sources implementing some complex authentication scheme might be an overkill. However, I think that a key thing with security is that if it's done, it has to be weel done. Somehow, a false sense of privacy (such as the cryptic URLs in easyDAS) might be a bad thing if the user is not aware of its weaknesses and end up unkwowingly sharing data deemed to remain private. So, unless a DAS sourceis really private and secure, it should be made clear that it is public (although maybe difficult to find). Bernat On Mon, Jan 16, 2012 at 18:10, Manuel Corpas wrote: > Hi guys, > > why not just trying to do something *really* easy that could be > implemented in a couple of weeks rather than trying to engineer the > perfect solution all at once? We don't really know how much demand > there is for authenticated DAS sources, so we may end up spending a > lot of effort for something that might be underused. > > The truth is that currently all DAS sources are public. Some "private > sources" available are those provided by easyDAS with its long > "cryptic" urls. That alone I think is already something. Any > improvement on the ability to keep sources private, whatever small, > would be a step in the right direction. > > My two cents. > > Manuel > > Manuel Corpas, PhD > Tel: +44.122349.2372 > Web: http://manuelcorpas.com/about/ > Twitter: @manuelcorpas > > > > On 16 January 2012 16:44, Jim Procter > wrote: > > Hi Andy, and anyone else still reading :) > > > > > > On 14/01/2012 01:03, Andy Jenkinson wrote: > >> > >> On 13 Jan 2012, at 13:04, Jim Procter wrote: > >> > >>> Hi. I see the dreaded secure DAS session topic as risen its head again. > >>> > >>> On 13/01/2012 00:42, Andy Jenkinson wrote: > >>>> > >>>> Originally I wanted to use a combination of OpenID and OAuth as an > >>>> end-to-end solution. However, OpenID is based around the expectation > that > >>>> you are authenticating with a website using a browser - the protocol > uses > >>>> HTTP redirects, and OpenID providers have to have some way of telling > you > >>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS > server > >>>> that needs to check that you are who you say you are, not just the > client. > >>>> For a client like Ensembl, your browser simply never communicates > with the > >>>> DAS server so the DAS server can't get you to authenticate with the > OpenID > >>>> provider. > >>> > >>> But Ensembl *does* need to know who you are in order to request data > that > >>> you are allowed access to. In the OAuth model, you would have to allow > >>> Ensembl to access privileged data from the third party DAS server, and > that > >>> would be achieved by the Ensembl browser presenting you with an OpenID > login > >>> and redirect to subsequent access control pages from the third party > server. > >> > >> Not sure I follow. > > > > Bother! am I feeding troll(s) again ? ;) > > > >> The problem with the above model isn't with Ensembl it's with the DAS > >> servers. DAS servers can't verify the user that is sitting at his > machine > >> (i.e. use OpenID) because they don't communicate with his machine. They > only > >> communicate, and can thus verify, the client (i.e. using OAuth). It > would be > >> better if the DAS server had some way to check the user directly (the > client > >> would be like a proxy for the user) but it is not possible with OpenID > under > >> Ensembl's current architecture (or at least it wasn't when I was > >> investigating). So that means you have to do one of the below options > >> instead (including having only Ensembl do the authentication). > > > > I'm not sure how different this is from the 3D security challenge from > your > > bank/credit card company that opens in an iframe after you put your > credit > > card details in to a secure shopping site and hit the confirm button. > > Authentication still has to happen in order for a transaction (in our > case > > one involving a DAS request/response session) between the DAS server and > the > > Ensembl server, before it renders the result in the page sent to your > > browser. > > > > As for dalliance and secure servers: > > > >> Basically yes, I would have thought there'd have to be some server-side > >> Dalliance script doing most of the work, with it hopefully being > possible to > >> pass the signed token to the browser so that the user's machine is > still the > >> one issuing the DAS request. > > > > it won't actually be happening that way - take a look at this: > > http://www.biodalliance.org/cors.html > > > >> However, I just saw this, which is a potential solution (for OAuth at > >> least): > >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ > >> I'm not sure it helps however, as we need to ensure that Dalliance will > >> only issue an authorised request to the DAS server if it has proof that > the > >> user has been authenticated and is on the access control list. With JS > being > >> editable on the client side, that check cannot be in JS. > > > > Interesting angle. I'm not actually sure what happens if the page from a > > server allowed to contact another server via CORS has been edited after > it > > arrived in the browser sandbox - but I don't think it is really that > much of > > an issue either way. > > > > There are two situations worth considering: > > 1. an honest user wants to access secure data. In which case, they must > > authenticate via the dalliance host page with servers providing secure > data > > in order to grant dalliance access via an authenticated cross-origin > request > > from a specified server on the trusted CORS list. > > 2. a black hat attempts to gain access to secure data. In which case they > > must fake an authentication. We then have to rely on the authentication > > framework detecting cracking attempts. > > > > The case when a (probably) honest user with valid credentials misuses the > > cross-origin request by messing with the javascript on the page is no > > different to having a user in a secure intranet who brings a virus in > via a > > physical device or over a VPN. Since DAS is rather limited in its scope > for > > exploits, I don't think there will be a problem with someone fiddling > with > > the javascript, assuming, of course, the servers hosting the > authenticated > > sources actually secured their server properly and have un-DOSable data > > providers at the back end... > > > >>>> Lastly, neither solution works for daemon-style clients (e.g. > >>>> command-line analysis applications where the user is not present), > again > >>>> because they can't use OpenID. The catch-all solution is to use X.509 > >>>> certificates (public/private key cryptography) but it is heavyweight > and > >>>> probably too complicated to provide a good user experience. Truth be > told, > >>>> it has proved difficult to discuss this topic amongst the community > because > >>>> it gets technically very complex. > >>> > >>> I'd be very much in favour of people who *need* to achieve this > spending > >>> some time during the developer days with an invited expert (with > special > >>> anti-trolling skills to counter folk like me) in a closed session, in > order > >>> to identify components that would enable both browser and non-browser > based > >>> clients to work with OAuth, and set up a trial OAuth DAS source > network for > >>> testing. There are libraries that support OAuth ( > http://oauth.net/code/) > >>> both for providers and consumers, but DAS client libraries will need > >>> extension to allow secure negotiation and signed DAS HTTP interaction, > and > >>> their APIs will need additional parameters/methods to allow session > >>> management. > >> > >> I have no objections to participating in principle, but I remain to > >> convinced it is worth the effort (i.e. that HTTP auth is insufficient > and > >> that OAuth would be adopted). > >> > > understood, and I agree, with the caveat that I'm not even sure how > familiar > > we all are with HTTP(s) auth styles, and personally, would appreciate it > if > > there was a practical session or tutorial on the ins and outs of > > authenticated DAS for those who need to do it. This could cover how > > different styles of authenticated DAS sources work with the existing > clients > > (e.g. do I really have to authenticate against every secure das source > every > > time a request is made?!). It could also involve a tutorial on setting > up a > > secure DAS source using various schemes. For example, a tutorial on SSO > > systems such as Shibboleth2 as used by the UK access management > federation > > could be useful if people needs to give specific UK academics access to a > > DAS source. > > > > Of course - nothing in this email constitutes me volunteering to give > such a > > tutorial .... > > > > Jim. > > > > _______________________________________________ > > DAS mailing list > > DAS at lists.open-bio.org > > http://lists.open-bio.org/mailman/listinfo/das > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das > From andy.jenkinson at ebi.ac.uk Mon Jan 16 18:31:52 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 16 Jan 2012 18:31:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: <4F145407.4000900@compbio.dundee.ac.uk> References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: I rather suspect this is a purely mental exercise, but that's fine for me ;) On 16 Jan 2012, at 16:44, Jim Procter wrote: > On 14/01/2012 01:03, Andy Jenkinson wrote: >> >> The problem with the above model isn't with Ensembl it's with the DAS servers. DAS servers can't verify the user that is sitting at his machine (i.e. use OpenID) because they don't communicate with his machine. They only communicate, and can thus verify, the client (i.e. using OAuth). It would be better if the DAS server had some way to check the user directly (the client would be like a proxy for the user) but it is not possible with OpenID under Ensembl's current architecture (or at least it wasn't when I was investigating). So that means you have to do one of the below options instead (including having only Ensembl do the authentication). > I'm not sure how different this is from the 3D security challenge from your bank/credit card company that opens in an iframe after you put your credit card details in to a secure shopping site and hit the confirm button. Authentication still has to happen in order for a transaction (in our case one involving a DAS request/response session) between the DAS server and the Ensembl server, before it renders the result in the page sent to your browser. It is similar, and the solutions we are discussing are similar. As I say, there are solutions but they require some coordination. > As for dalliance and secure servers: >> Basically yes, I would have thought there'd have to be some server-side Dalliance script doing most of the work, with it hopefully being possible to pass the signed token to the browser so that the user's machine is still the one issuing the DAS request. > it won't actually be happening that way - take a look at this: http://www.biodalliance.org/cors.html CORS is what allows Dalliance to avoid having to communicate with DAS servers using a server in the middle (as Ensembl does). I'm saying that if a server is required to do the authentication bit (as I suspect it would be), I hope there is no technical block to having the user's computer send the signed token to the DAS server (i.e. via CORS) rather than the "Dalliance authenticator proxy". It is a slightly different model so I can't say it would definitely work. >> However, I just saw this, which is a potential solution (for OAuth at least): >> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >> I'm not sure it helps however, as we need to ensure that Dalliance will only issue an authorised request to the DAS server if it has proof that the user has been authenticated and is on the access control list. With JS being editable on the client side, that check cannot be in JS. > Interesting angle. I'm not actually sure what happens if the page from a server allowed to contact another server via CORS has been edited after it arrived in the browser sandbox - but I don't think it is really that much of an issue either way. CORS really isn't an issue - the server either allows CORS or it doesn't (in which case it is not supported by Dalliance) - it does not interact with the security side of things in any problematic way. You can use HTTP authentication in combination with CORS just fine, but we are discussing OpenID + OAuth. I am assuming that CORS is enabled on the DAS server because if it is not then Dalliance will not work. What I am talking about is the fact that Dalliance is a JS application. That is, all its code is completely editable by anyone who cares to open the Developer console, Firebug etc. There is no advantage to using OpenID+OAuth over HTTP authentication (which is what I have implemented in ProServer already so it works with Dalliance as-is) if the user can just disable the bit of the code that authenticates you. Thus we would have to ensure that the Dalliance JS only gets access to an OAuth token if an externally verifiable set of conditions is met. This is what, for me, means that there must be a server component rather than it all being done in JS. > There are two situations worth considering: > 1. an honest user wants to access secure data. In which case, they must authenticate via the dalliance host page with servers providing secure data in order to grant dalliance access via an authenticated cross-origin request from a specified server on the trusted CORS list. > 2. a black hat attempts to gain access to secure data. In which case they must fake an authentication. We then have to rely on the authentication framework detecting cracking attempts. > > The case when a (probably) honest user with valid credentials misuses the cross-origin request by messing with the javascript on the page is no different to having a user in a secure intranet who brings a virus in via a physical device or over a VPN. Since DAS is rather limited in its scope for exploits, I don't think there will be a problem with someone fiddling with the javascript, assuming, of course, the servers hosting the authenticated sources actually secured their server properly and have un-DOSable data providers at the back end... I think we're talking crossed purposes. CORS is not a way of securing servers, it's just a way of lifting the restriction placed on Javascript by browsers that it can't request URLs outside the domain of the webpage the Javascript came from. An authenticated cross-origin request (in CORS language) simply means a regular HTTP digest or HTTP basic authenticated request that happens to be setting the headers that are needed to get around this restriction. The potential for misuse is not from misusing the CORS request (indeed interfering with CORS is not possible by fiddling with the javascript). A cross-origin model is completely standard for DAS, indeed it is the whole point of having a distributed system. Nor is the misuse I am worried about to do with faking an authentication. It is simply making a trivial change in a trusted client to manipulate it into accessing secured data by bypassing the need to do the authentication step at all. OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. >>> I'd be very much in favour of people who *need* to achieve this spending some time during the developer days with an invited expert (with special anti-trolling skills to counter folk like me) in a closed session, in order to identify components that would enable both browser and non-browser based clients to work with OAuth, and set up a trial OAuth DAS source network for testing. There are libraries that support OAuth (http://oauth.net/code/) both for providers and consumers, but DAS client libraries will need extension to allow secure negotiation and signed DAS HTTP interaction, and their APIs will need additional parameters/methods to allow session management. >> I have no objections to participating in principle, but I remain to convinced it is worth the effort (i.e. that HTTP auth is insufficient and that OAuth would be adopted). >> > understood, and I agree, with the caveat that I'm not even sure how familiar we all are with HTTP(s) auth styles, and personally, would appreciate it if there was a practical session or tutorial on the ins and outs of authenticated DAS for those who need to do it. This could cover how different styles of authenticated DAS sources work with the existing clients (e.g. do I really have to authenticate against every secure das source every time a request is made?!). It could also involve a tutorial on setting up a secure DAS source using various schemes. For example, a tutorial on SSO systems such as Shibboleth2 as used by the UK access management federation could be useful if people needs to give specific UK academics access to a DAS source. > > Of course - nothing in this email constitutes me volunteering to give such a tutorial .... > Jim. From andy.jenkinson at ebi.ac.uk Mon Jan 16 18:34:51 2012 From: andy.jenkinson at ebi.ac.uk (Andy Jenkinson) Date: Mon, 16 Jan 2012 18:34:51 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: I agree with this. Security through obscurity should never be presented as "private". We should continue to implement some form of authentication but we should keep it simple to start with. On 16 Jan 2012, at 17:20, Bernat Gel wrote: > Hi all, > > I agree with Manuel (and I think Andy) that without a real demand for > private DAS sources implementing some complex authentication scheme might > be an overkill. However, I think that a key thing with security is that if > it's done, it has to be weel done. Somehow, a false sense of privacy (such > as the cryptic URLs in easyDAS) might be a bad thing if the user is not > aware of its weaknesses and end up unkwowingly sharing data deemed to > remain private. > > So, unless a DAS sourceis really private and secure, it should be made > clear that it is public (although maybe difficult to find). > > Bernat > > On Mon, Jan 16, 2012 at 18:10, Manuel Corpas wrote: > >> Hi guys, >> >> why not just trying to do something *really* easy that could be >> implemented in a couple of weeks rather than trying to engineer the >> perfect solution all at once? We don't really know how much demand >> there is for authenticated DAS sources, so we may end up spending a >> lot of effort for something that might be underused. >> >> The truth is that currently all DAS sources are public. Some "private >> sources" available are those provided by easyDAS with its long >> "cryptic" urls. That alone I think is already something. Any >> improvement on the ability to keep sources private, whatever small, >> would be a step in the right direction. >> >> My two cents. >> >> Manuel >> >> Manuel Corpas, PhD >> Tel: +44.122349.2372 >> Web: http://manuelcorpas.com/about/ >> Twitter: @manuelcorpas >> >> >> >> On 16 January 2012 16:44, Jim Procter >> wrote: >>> Hi Andy, and anyone else still reading :) >>> >>> >>> On 14/01/2012 01:03, Andy Jenkinson wrote: >>>> >>>> On 13 Jan 2012, at 13:04, Jim Procter wrote: >>>> >>>>> Hi. I see the dreaded secure DAS session topic as risen its head again. >>>>> >>>>> On 13/01/2012 00:42, Andy Jenkinson wrote: >>>>>> >>>>>> Originally I wanted to use a combination of OpenID and OAuth as an >>>>>> end-to-end solution. However, OpenID is based around the expectation >> that >>>>>> you are authenticating with a website using a browser - the protocol >> uses >>>>>> HTTP redirects, and OpenID providers have to have some way of telling >> you >>>>>> are logged in - cookies, forms etc. Ideally in DAS, it is the DAS >> server >>>>>> that needs to check that you are who you say you are, not just the >> client. >>>>>> For a client like Ensembl, your browser simply never communicates >> with the >>>>>> DAS server so the DAS server can't get you to authenticate with the >> OpenID >>>>>> provider. >>>>> >>>>> But Ensembl *does* need to know who you are in order to request data >> that >>>>> you are allowed access to. In the OAuth model, you would have to allow >>>>> Ensembl to access privileged data from the third party DAS server, and >> that >>>>> would be achieved by the Ensembl browser presenting you with an OpenID >> login >>>>> and redirect to subsequent access control pages from the third party >> server. >>>> >>>> Not sure I follow. >>> >>> Bother! am I feeding troll(s) again ? ;) >>> >>>> The problem with the above model isn't with Ensembl it's with the DAS >>>> servers. DAS servers can't verify the user that is sitting at his >> machine >>>> (i.e. use OpenID) because they don't communicate with his machine. They >> only >>>> communicate, and can thus verify, the client (i.e. using OAuth). It >> would be >>>> better if the DAS server had some way to check the user directly (the >> client >>>> would be like a proxy for the user) but it is not possible with OpenID >> under >>>> Ensembl's current architecture (or at least it wasn't when I was >>>> investigating). So that means you have to do one of the below options >>>> instead (including having only Ensembl do the authentication). >>> >>> I'm not sure how different this is from the 3D security challenge from >> your >>> bank/credit card company that opens in an iframe after you put your >> credit >>> card details in to a secure shopping site and hit the confirm button. >>> Authentication still has to happen in order for a transaction (in our >> case >>> one involving a DAS request/response session) between the DAS server and >> the >>> Ensembl server, before it renders the result in the page sent to your >>> browser. >>> >>> As for dalliance and secure servers: >>> >>>> Basically yes, I would have thought there'd have to be some server-side >>>> Dalliance script doing most of the work, with it hopefully being >> possible to >>>> pass the signed token to the browser so that the user's machine is >> still the >>>> one issuing the DAS request. >>> >>> it won't actually be happening that way - take a look at this: >>> http://www.biodalliance.org/cors.html >>> >>>> However, I just saw this, which is a potential solution (for OAuth at >>>> least): >>>> http://derek.io/blog/2010/how-to-secure-oauth-in-javascript/ >>>> I'm not sure it helps however, as we need to ensure that Dalliance will >>>> only issue an authorised request to the DAS server if it has proof that >> the >>>> user has been authenticated and is on the access control list. With JS >> being >>>> editable on the client side, that check cannot be in JS. >>> >>> Interesting angle. I'm not actually sure what happens if the page from a >>> server allowed to contact another server via CORS has been edited after >> it >>> arrived in the browser sandbox - but I don't think it is really that >> much of >>> an issue either way. >>> >>> There are two situations worth considering: >>> 1. an honest user wants to access secure data. In which case, they must >>> authenticate via the dalliance host page with servers providing secure >> data >>> in order to grant dalliance access via an authenticated cross-origin >> request >>> from a specified server on the trusted CORS list. >>> 2. a black hat attempts to gain access to secure data. In which case they >>> must fake an authentication. We then have to rely on the authentication >>> framework detecting cracking attempts. >>> >>> The case when a (probably) honest user with valid credentials misuses the >>> cross-origin request by messing with the javascript on the page is no >>> different to having a user in a secure intranet who brings a virus in >> via a >>> physical device or over a VPN. Since DAS is rather limited in its scope >> for >>> exploits, I don't think there will be a problem with someone fiddling >> with >>> the javascript, assuming, of course, the servers hosting the >> authenticated >>> sources actually secured their server properly and have un-DOSable data >>> providers at the back end... >>> >>>>>> Lastly, neither solution works for daemon-style clients (e.g. >>>>>> command-line analysis applications where the user is not present), >> again >>>>>> because they can't use OpenID. The catch-all solution is to use X.509 >>>>>> certificates (public/private key cryptography) but it is heavyweight >> and >>>>>> probably too complicated to provide a good user experience. Truth be >> told, >>>>>> it has proved difficult to discuss this topic amongst the community >> because >>>>>> it gets technically very complex. >>>>> >>>>> I'd be very much in favour of people who *need* to achieve this >> spending >>>>> some time during the developer days with an invited expert (with >> special >>>>> anti-trolling skills to counter folk like me) in a closed session, in >> order >>>>> to identify components that would enable both browser and non-browser >> based >>>>> clients to work with OAuth, and set up a trial OAuth DAS source >> network for >>>>> testing. There are libraries that support OAuth ( >> http://oauth.net/code/) >>>>> both for providers and consumers, but DAS client libraries will need >>>>> extension to allow secure negotiation and signed DAS HTTP interaction, >> and >>>>> their APIs will need additional parameters/methods to allow session >>>>> management. >>>> >>>> I have no objections to participating in principle, but I remain to >>>> convinced it is worth the effort (i.e. that HTTP auth is insufficient >> and >>>> that OAuth would be adopted). >>>> >>> understood, and I agree, with the caveat that I'm not even sure how >> familiar >>> we all are with HTTP(s) auth styles, and personally, would appreciate it >> if >>> there was a practical session or tutorial on the ins and outs of >>> authenticated DAS for those who need to do it. This could cover how >>> different styles of authenticated DAS sources work with the existing >> clients >>> (e.g. do I really have to authenticate against every secure das source >> every >>> time a request is made?!). It could also involve a tutorial on setting >> up a >>> secure DAS source using various schemes. For example, a tutorial on SSO >>> systems such as Shibboleth2 as used by the UK access management >> federation >>> could be useful if people needs to give specific UK academics access to a >>> DAS source. >>> >>> Of course - nothing in this email constitutes me volunteering to give >> such a >>> tutorial .... >>> >>> Jim. >>> >>> _______________________________________________ >>> DAS mailing list >>> DAS at lists.open-bio.org >>> http://lists.open-bio.org/mailman/listinfo/das >> >> _______________________________________________ >> DAS mailing list >> DAS at lists.open-bio.org >> http://lists.open-bio.org/mailman/listinfo/das >> > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das From dan.bolser at gmail.com Mon Jan 16 21:12:52 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Mon, 16 Jan 2012 21:12:52 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: On 16 January 2012 18:31, Andy Jenkinson wrote: > I rather suspect this is a purely mental exercise, but that's fine for me ;) > OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. Right, the person who 'owns' the data (i.e. a list of contacts hosted on a Google account) explicitly grants the third party 'app' permission to access the data (via the account) in a specified way (as defined by the Google APIs). That app can then email all your contacts in the middle of the night while you're sleeping, but you trust that that won't happen when you click the 'grant' button. i.e. I (the verified me) can grant Ensembl permission to access my SNP genotype data from 23andMe (hah), and I trust Ensemble not to do anything nasty with that data when I log off. Although it's a bit of a pain to set this up, the point is that literally thousands of app developers (including me) have done it before, and there are hundreds of docs. Here is where I started when I built a command line twitter bot: https://dev.twitter.com/docs/auth I'm not trying to say its quick and easy to do and everyone should do it like this, I just thought I'd provide the above encapsulation, which hopefully isn't too far from how it could be done. Cheers, Dan. From dan.bolser at gmail.com Mon Jan 16 23:16:57 2012 From: dan.bolser at gmail.com (Dan Bolser) Date: Mon, 16 Jan 2012 23:16:57 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: Heh... it doesn't work: https://github.com/dbolser/TwitterBot---nowlistening-Perl-script-for-xmms Not sure what I'm doing wrong. On 16 January 2012 21:12, Dan Bolser wrote: > On 16 January 2012 18:31, Andy Jenkinson wrote: >> I rather suspect this is a purely mental exercise, but that's fine for me ;) > > > >> OAuth is entirely based upon the notion that the server with the data (e.g. Google Contacts) can trust the application (e.g. the Android Contacts app) to do the right thing with the data. There is no requirement that the person who owns the data, or any other person, has to be present, and the application doesn't have to prove that this will happen. It just has to get the user to agree that the application can be trusted. It's up to us therefore to provide a secure link between OpenID and OAuth. > > Right, the person who 'owns' the data (i.e. a list of contacts hosted > on a Google account) explicitly grants the third party 'app' > permission to access the data (via the account) in a specified way (as > defined by the Google APIs). That app can then email all your contacts > in the middle of the night while you're sleeping, but you trust that > that won't happen when you click the 'grant' button. > > i.e. I (the verified me) can grant Ensembl permission to access my SNP > genotype data from 23andMe (hah), and I trust Ensemble not to do > anything nasty with that data when I log off. > > Although it's a bit of a pain to set this up, the point is that > literally thousands of app developers (including me) have done it > before, and there are hundreds of docs. Here is where I started when I > built a command line twitter bot: > https://dev.twitter.com/docs/auth > > > I'm not trying to say its quick and easy to do and everyone should do > it like this, I just thought I'd provide the above encapsulation, > which hopefully isn't too far from how it could be done. > > > Cheers, > Dan. From jw12 at sanger.ac.uk Tue Jan 17 10:44:40 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Tue, 17 Jan 2012 10:44:40 +0000 Subject: [DAS] Personal genomics/Deploying a DAS server for Dummies/6 Easy steps In-Reply-To: References: <5432FAF3-7483-4007-8BB1-CC53B016A851@sanger.ac.uk> <3B01C973-ACA3-4A50-9D6B-9B64072E3556@sanger.ac.uk> <806E3F13-2037-439B-8DCC-543F4E4908A6@ebi.ac.uk> <25B5FE93-D36B-4E6B-ACDE-BDD9C05F9BF5@ebi.ac.uk> <4F102BEB.1020101@compbio.dundee.ac.uk> <8C86E6FB-9157-4F0C-91CC-CE87E8468C50@ebi.ac.uk> <4F145407.4000900@compbio.dundee.ac.uk> Message-ID: <5867A14E-E3D8-4E9A-94F5-83AFEA60FF2F@sanger.ac.uk> Hi I'd be against using OAuth because it will probably suffer from the same limitations that OpenId suffered from. 1) It will be the in thing for a while with support for certain programming languages or environments (such as browsers) but will soon go out of fashion when something else with a cool sounding name comes along :) 2) It takes us away even more than we already are from the principle that the DAS system should be simple. 3) If it's complicated it can be difficult to change your application if the environment that you are running your application in changes or other factors make you change it (e.g. when the sanger web site no longer supported sticky sessions I had to try and rewrite the openID code for the registry which turned out to be dependent on normal in memory Java sessions, so I had to change code that I didn't really understand, thus introducing security holes, so I ended up turning to good old username and passwords). 4) Everybody understands username and passwords boxes in an user interface whereas if we start going on about OAuth authentication and using their facebook accounts people will get frightened off - I'm also not sure I won't my gmail or facebook accounts to log me in to the DAS registry automatically and start distributing my contact lists ;) My preference and I think most other peoples when we have been on this discussion before (3rd time around??) is to use basic https security that has been around for years. I also prefer the option of sending the username and passwords for every request for a secure data source as the server then remains in one state (the registry sends them in the headers, MyDAS currently sends them in the xml - we need to decide which to use). This would be in preference to an amazon S3 type system where tokens are given for a limited period of time to the client/user (the S3 type system would be my second option). These would be a good balance between ease of development/understanding and security. If specific clients and servers wanted to add extra security they could add IP restrictions so they know the client is hosted at a specific place. Cheers Jonathan. On 16 Jan 2012, at 23:16, Dan Bolser wrote: > Heh... it doesn't work: > > https://github.com/dbolser/TwitterBot---nowlistening-Perl-script-for-xmms > > Not sure what I'm doing wrong. > > > On 16 January 2012 21:12, Dan Bolser wrote: >> On 16 January 2012 18:31, Andy Jenkinson >> wrote: >>> I rather suspect this is a purely mental exercise, but that's fine >>> for me ;) >> >> >> >>> OAuth is entirely based upon the notion that the server with the >>> data (e.g. Google Contacts) can trust the application (e.g. the >>> Android Contacts app) to do the right thing with the data. There >>> is no requirement that the person who owns the data, or any other >>> person, has to be present, and the application doesn't have to >>> prove that this will happen. It just has to get the user to agree >>> that the application can be trusted. It's up to us therefore to >>> provide a secure link between OpenID and OAuth. >> >> Right, the person who 'owns' the data (i.e. a list of contacts hosted >> on a Google account) explicitly grants the third party 'app' >> permission to access the data (via the account) in a specified way >> (as >> defined by the Google APIs). That app can then email all your >> contacts >> in the middle of the night while you're sleeping, but you trust that >> that won't happen when you click the 'grant' button. >> >> i.e. I (the verified me) can grant Ensembl permission to access my >> SNP >> genotype data from 23andMe (hah), and I trust Ensemble not to do >> anything nasty with that data when I log off. >> >> Although it's a bit of a pain to set this up, the point is that >> literally thousands of app developers (including me) have done it >> before, and there are hundreds of docs. Here is where I started >> when I >> built a command line twitter bot: >> https://dev.twitter.com/docs/auth >> >> >> I'm not trying to say its quick and easy to do and everyone should do >> it like this, I just thought I'd provide the above encapsulation, >> which hopefully isn't too far from how it could be done. >> >> >> Cheers, >> Dan. > > _______________________________________________ > DAS mailing list > DAS at lists.open-bio.org > http://lists.open-bio.org/mailman/listinfo/das Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE. From jw12 at sanger.ac.uk Fri Jan 20 10:41:18 2012 From: jw12 at sanger.ac.uk (Jonathan Warren) Date: Fri, 20 Jan 2012 10:41:18 +0000 Subject: [DAS] DAS Workshop Registrations Message-ID: <0F5CE38F-8FC4-4AEC-9BC4-6EB76BCA4850@sanger.ac.uk> Hi If you are intending to come to the DAS workshop this year can you register soon as we would like to get an idea of numbers for the 3 respective days (1st day tutorials and short talks, 2nd and 3rd day developers hackathon). This will then enable us to start firming up the schedule for the 3 days. If you are not intending to come to the workshop can I ask for some feedback that may help us entice you in the future? Many thanks Jonathan. See workshop email for you convenience below: DAS is currently being used to share annotations on genomes, protein alignments, structural and interaction information. If you are interested in sharing biological information the DAS workshop below may be of interest to you. Registration is open for the 2012 DAS workshop (27-29 February) at the Genome Campus, Hinxton UK. If you are interested in attending, please find out more by going to http://www.ebi.ac.uk/training/onsite/120227_DAS.html and register via the web link at the bottom of the page. This workshop will cater for novice to expert DAS users as each day is optional. Please register early as places will be limited. Registration closes 10 February 2012 - 12:00. If you are interested in giving a 15 minute talk on the second day please email Jonathan Warren using jonathan.warren at sanger.ac.uk Many thanks The Sanger/EBI DAS team. Jonathan Warren Senior Developer and DAS coordinator blog: http://biodasman.wordpress.com/ jw12 at sanger.ac.uk Ext: 2314 Telephone: 01223 492314 -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.