From Richard.Holland at agresearch.co.nz Mon Jan 5 22:26:50 2004 From: Richard.Holland at agresearch.co.nz (Holland, Richard) Date: Mon Jan 5 23:05:00 2004 Subject: [DAS] RE: DAS security Message-ID: Hi, Jonathan told me about this problem of restricting access to internal DAS sources by IP address in Ensembl, and I made a custom solution to it that we have been testing for about 2 months now, and appears to work just fine. Note that this is in addition to removing the code from Ensembl that allows users to add their own external DAS tracks - I'll leave that to Jonathan to explain. The solution works at the Ensembl level and not the DAS source level, but this is sufficient if you restrict the DAS server firewall to only allow connections to DAS from the web server that is running Ensembl. An explanation of how it works will follow after these instructions, but first there are a number of things that must be done to make it all work (ENSEMBL_HOME refers to the location of your Ensembl installation): 1. Think up some security groups for internal DAS sources to belong to. For instance, all human sources could belong to a group labelled 'human'. Sources can only belong to one group. 2. In ENSEMBL_HOME/conf/<>.ini (eg. Homo_sapiens.ini), define all your permanent internal DAS sources as normal in the DAS CONFIG section. Make sure you list all sources you want here regardless of groups, as users will not be able to add their own if you have implemented Jonathan's modifications as mentioned above. Now, for each DAS source, add a line reading: group = human ...or whatever group you want the DAS source to belong to. This option goes in with all the others, eg. linktext, linkURL, depth, strand etc. 3. Create a file called ENSEMBL_HOME/conf/Security.pm . Give it the same permissions as the other .pm scripts in ENSEMBL_HOME/conf. It should contain the following Perl code: ---BEGIN CODE--- #!/usr/local/bin/perl package Security; use strict; # Defines one function which takes a security group as a parameter, # and returns 1 if the current user is allowed to see that track, 0 if not. # The constructor sets the username. # --- Richard Holland, AgResearch, January 2004 # email richard at uwc dot net sub new { my $class = shift; my $user = shift; bless ({'_user'=>$user},$class); } sub checkSecurity { my $self = shift; my $securitygroup = shift; my $securityuser = $self->{'_user'}; # Allow access if no group restriction specified return 1 if not defined $securitygroup; # Block access if group restriction present and username not given return 0 if not defined $securityuser; # Use some kind of authentication here to look up the securitygroup value # against the securityuser value and determine if the user is allowed to see # this group. This is entirely customisable - this example just returns 1 to # say that permission is granted. The code should return 0 if permission is # denied. In our local installation, there is a call to a web service here using # SOAP::Lite which does the lookup for us. return 1; # Default to permission granted - do not leave this here! } 1; ---END CODE--- 4. Configure Apache to require authentication on the ENSEMBL_HOME/perl folder - the usual way to do this is by .htaccess and .htpasswd but it is up to you, as long as it requires authentication and can provide a named user for the CGI remote_user parameter. 5. Edit ENSEMBL_HOME/perl/contigview and find the line (right near the top) that reads: my $uca = new EnsEMBL::Web::UserConfigAdaptor( EnsWeb::species_defs->SITE_TYPE ); Insert the following code directly after it: ---BEGIN CODE--- # BEGIN ENSEMBL SECURITY PATCH use Security; my $remote_user = $cgi->remote_user; my $security = new Security($remote_user); my $dassources = EnsWeb::species_defs->ENSEMBL_INTERNAL_DAS_SOURCES; foreach (keys %$dassources) { if ($security->checkSecurity($dassources->{$_}->{'security'}) == 0) { warn "...access DENIED to DAS source $_ for ".($remote_user||''); delete $dassources->{$_}; } else { warn "...access granted to DAS source $_ for ".($remote_user||''); } } EnsWeb::species_defs->set_config($ENV{'ENSEMBL_SPECIES'},'ENSEMBL_INTERN AL_DAS_SOURCES',$dassources,1); # END ENSEMBL SECURITY PATCH ---END CODE--- 6. That's it! Once you have implemented some kind of authentication in the Security.pm module from step, you will find that access restriction works. Now, the explanation. By adding the group parameter to the internal DAS sources in the species configuration file, we can categorise DAS sources. By adding the Apache authentication to the perl folder, we can get a username. Hence, we can limit sources by username by looking at the group the source belongs to. The code assumes that if no group has been given in the config file, then the source can be viewed by anybody. If a group is given but the user is unauthenticated (if you get .htaccess wrong for example), then access is denied to that source. If both a group and a username are present, then the last part of Security.pm is reached, which performs the authentication lookup. In our local installation we have a database of group names and usernames which we look up via a quick web service query, but any mechanism could be used. The mechanism of the Security.pm routine is simple - return 1 if permission is granted, return 0 if not. The alteration to contigview simply waits for the DAS source configuration to be loaded, then loops through the internal DAS sources and gets their group names. It then passes the group name to the Security.pm module for checking. If permission is granted, no further action is taken. If it is not granted, then the external DAS source is completely deleted from the configuration hash, so that the rest of Ensembl never knew it even existed. Of course this happens only in the memory copy of the config - the config on disk is left untouched. You can see who is granted or denied by looking at the Ensembl logs - hence you can bugfix your allocation of groups and usernames. cheers, Richard --- Richard Holland Bioinformatics Database Developer ITS, Agresearch Invermay x3279 -----Original Message----- From: Tony Cox [mailto:avc@sanger.ac.uk] Sent: Tuesday, 23 September 2003 9:59 p.m. To: Warren, Jonathan Cc: ensembl-dev@ebi.ac.uk; Subject: Re: DAS security On Tue, 23 Sep 2003, Warren, Jonathan wrote: +>Hi +> +>We have ensembl and Dazzle installed locally and have internal IP +>issues where some groups are not supposed to be allowed to see certain +>tracks. There is no built in security model in Ensembl (although this is not to say Ensembl is built without regard to security!) Since it has always been an open source/data project we have not engineered a system for hiding some data form a subset of users. I think you would have to use either a proxy layer to filter data by IP address ranges or else you will have to embed some IP-based track switching in the drawing code. We'd be very interested to know if you come up with a nice solution! regards Tony +>Where is the best place to change the ensembl code to restrict access +>to only certain DSNs (if a change in the code is needed?). Is there +>one place that would stop people using the "Manage Sources" menu and +>would allow us to restrict the DAS sources currently configured in our +>HomoSapiens.ini file to certain users? +> +>Cheers +> +>Jonathan +> +> +> +>====================================================================== +>= +>Attention: The information contained in this message and/or attachments +>from AgResearch Limited is intended only for the persons or entities +>to which it is addressed and may contain confidential and/or privileged +>material. Any review, retransmission, dissemination or other use of, or +>taking of any action in reliance upon, this information by persons or +>entities other than the intended recipients is prohibited by AgResearch +>Limited. If you have received this message in error, please notify the +>sender immediately. +>====================================================================== = +> ****************************************************** Tony Cox Email:avc@sanger.ac.uk Sanger Institute WWW:www.sanger.ac.uk Wellcome Trust Genome Campus Head,Software Services Hinxton Tel: +44 1223 834244 Cambs. CB10 1SA Fax: +44 1223 494919 ****************************************************** ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= From Jonathan.Warren at agresearch.co.nz Mon Jan 5 23:14:00 2004 From: Jonathan.Warren at agresearch.co.nz (Warren, Jonathan) Date: Mon Jan 5 23:53:01 2004 Subject: [DAS] RE: DAS security Message-ID: No rocket science- I just removed: ../ensembl/perl/default/externaldas As our users can't configure their own external DAS sources anyway due to very strict security here, I could remove the whole thing so the "manage DAS sources" menu in contigview doesn't work. We should set up a more sensible web page to come up as apposed to the current error message though. Cheers Jonathan. -----Original Message----- From: Holland, Richard Sent: Tuesday, 6 January 2004 4:27 p.m. To: Tony Cox; Warren, Jonathan Cc: ensembl-dev@ebi.ac.uk; das@biodas.org Subject: RE: DAS security Hi, Jonathan told me about this problem of restricting access to internal DAS sources by IP address in Ensembl, and I made a custom solution to it that we have been testing for about 2 months now, and appears to work just fine. Note that this is in addition to removing the code from Ensembl that allows users to add their own external DAS tracks - I'll leave that to Jonathan to explain. The solution works at the Ensembl level and not the DAS source level, but this is sufficient if you restrict the DAS server firewall to only allow connections to DAS from the web server that is running Ensembl. An explanation of how it works will follow after these instructions, but first there are a number of things that must be done to make it all work (ENSEMBL_HOME refers to the location of your Ensembl installation): 1. Think up some security groups for internal DAS sources to belong to. For instance, all human sources could belong to a group labelled 'human'. Sources can only belong to one group. 2. In ENSEMBL_HOME/conf/<>.ini (eg. Homo_sapiens.ini), define all your permanent internal DAS sources as normal in the DAS CONFIG section. Make sure you list all sources you want here regardless of groups, as users will not be able to add their own if you have implemented Jonathan's modifications as mentioned above. Now, for each DAS source, add a line reading: group = human ...or whatever group you want the DAS source to belong to. This option goes in with all the others, eg. linktext, linkURL, depth, strand etc. 3. Create a file called ENSEMBL_HOME/conf/Security.pm . Give it the same permissions as the other .pm scripts in ENSEMBL_HOME/conf. It should contain the following Perl code: ---BEGIN CODE--- #!/usr/local/bin/perl package Security; use strict; # Defines one function which takes a security group as a parameter, # and returns 1 if the current user is allowed to see that track, 0 if not. # The constructor sets the username. # --- Richard Holland, AgResearch, January 2004 # email richard at uwc dot net sub new { my $class = shift; my $user = shift; bless ({'_user'=>$user},$class); } sub checkSecurity { my $self = shift; my $securitygroup = shift; my $securityuser = $self->{'_user'}; # Allow access if no group restriction specified return 1 if not defined $securitygroup; # Block access if group restriction present and username not given return 0 if not defined $securityuser; # Use some kind of authentication here to look up the securitygroup value # against the securityuser value and determine if the user is allowed to see # this group. This is entirely customisable - this example just returns 1 to # say that permission is granted. The code should return 0 if permission is # denied. In our local installation, there is a call to a web service here using # SOAP::Lite which does the lookup for us. return 1; # Default to permission granted - do not leave this here! } 1; ---END CODE--- 4. Configure Apache to require authentication on the ENSEMBL_HOME/perl folder - the usual way to do this is by .htaccess and .htpasswd but it is up to you, as long as it requires authentication and can provide a named user for the CGI remote_user parameter. 5. Edit ENSEMBL_HOME/perl/contigview and find the line (right near the top) that reads: my $uca = new EnsEMBL::Web::UserConfigAdaptor( EnsWeb::species_defs->SITE_TYPE ); Insert the following code directly after it: ---BEGIN CODE--- # BEGIN ENSEMBL SECURITY PATCH use Security; my $remote_user = $cgi->remote_user; my $security = new Security($remote_user); my $dassources = EnsWeb::species_defs->ENSEMBL_INTERNAL_DAS_SOURCES; foreach (keys %$dassources) { if ($security->checkSecurity($dassources->{$_}->{'security'}) == 0) { warn "...access DENIED to DAS source $_ for ".($remote_user||''); delete $dassources->{$_}; } else { warn "...access granted to DAS source $_ for ".($remote_user||''); } } EnsWeb::species_defs->set_config($ENV{'ENSEMBL_SPECIES'},'ENSEMBL_INTERN AL_DAS_SOURCES',$dassources,1); # END ENSEMBL SECURITY PATCH ---END CODE--- 6. That's it! Once you have implemented some kind of authentication in the Security.pm module from step, you will find that access restriction works. Now, the explanation. By adding the group parameter to the internal DAS sources in the species configuration file, we can categorise DAS sources. By adding the Apache authentication to the perl folder, we can get a username. Hence, we can limit sources by username by looking at the group the source belongs to. The code assumes that if no group has been given in the config file, then the source can be viewed by anybody. If a group is given but the user is unauthenticated (if you get .htaccess wrong for example), then access is denied to that source. If both a group and a username are present, then the last part of Security.pm is reached, which performs the authentication lookup. In our local installation we have a database of group names and usernames which we look up via a quick web service query, but any mechanism could be used. The mechanism of the Security.pm routine is simple - return 1 if permission is granted, return 0 if not. The alteration to contigview simply waits for the DAS source configuration to be loaded, then loops through the internal DAS sources and gets their group names. It then passes the group name to the Security.pm module for checking. If permission is granted, no further action is taken. If it is not granted, then the external DAS source is completely deleted from the configuration hash, so that the rest of Ensembl never knew it even existed. Of course this happens only in the memory copy of the config - the config on disk is left untouched. You can see who is granted or denied by looking at the Ensembl logs - hence you can bugfix your allocation of groups and usernames. cheers, Richard --- Richard Holland Bioinformatics Database Developer ITS, Agresearch Invermay x3279 -----Original Message----- From: Tony Cox [mailto:avc@sanger.ac.uk] Sent: Tuesday, 23 September 2003 9:59 p.m. To: Warren, Jonathan Cc: ensembl-dev@ebi.ac.uk; Subject: Re: DAS security On Tue, 23 Sep 2003, Warren, Jonathan wrote: +>Hi +> +>We have ensembl and Dazzle installed locally and have internal IP +>issues where some groups are not supposed to be allowed to see certain +>tracks. There is no built in security model in Ensembl (although this is not to say Ensembl is built without regard to security!) Since it has always been an open source/data project we have not engineered a system for hiding some data form a subset of users. I think you would have to use either a proxy layer to filter data by IP address ranges or else you will have to embed some IP-based track switching in the drawing code. We'd be very interested to know if you come up with a nice solution! regards Tony +>Where is the best place to change the ensembl code to restrict access +>to only certain DSNs (if a change in the code is needed?). Is there +>one place that would stop people using the "Manage Sources" menu and +>would allow us to restrict the DAS sources currently configured in our +>HomoSapiens.ini file to certain users? +> +>Cheers +> +>Jonathan +> +> +> +>====================================================================== +>= +>Attention: The information contained in this message and/or attachments +>from AgResearch Limited is intended only for the persons or entities +>to which it is addressed and may contain confidential and/or privileged +>material. Any review, retransmission, dissemination or other use of, or +>taking of any action in reliance upon, this information by persons or +>entities other than the intended recipients is prohibited by AgResearch +>Limited. If you have received this message in error, please notify the +>sender immediately. +>====================================================================== = +> ****************************************************** Tony Cox Email:avc@sanger.ac.uk Sanger Institute WWW:www.sanger.ac.uk Wellcome Trust Genome Campus Head,Software Services Hinxton Tel: +44 1223 834244 Cambs. CB10 1SA Fax: +44 1223 494919 ****************************************************** ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= From Richard.Holland at agresearch.co.nz Tue Jan 6 00:12:23 2004 From: Richard.Holland at agresearch.co.nz (Holland, Richard) Date: Tue Jan 6 00:20:08 2004 Subject: [DAS] RE: DAS security Message-ID: Just noticed I made a mistake here - the references to the 'group' parameter should in fact refer to the 'security' parameter - ie. security = human, not group = human. Sorry about that! cheers, Richard -----Original Message----- From: Warren, Jonathan Sent: Tue 1/6/2004 5:14 PM To: Holland, Richard; 'Tony Cox' Cc: 'ensembl-dev@ebi.ac.uk'; 'das@biodas.org' Subject: RE: DAS security No rocket science- I just removed: ../ensembl/perl/default/externaldas As our users can't configure their own external DAS sources anyway due to very strict security here, I could remove the whole thing so the "manage DAS sources" menu in contigview doesn't work. We should set up a more sensible web page to come up as apposed to the current error message though. Cheers Jonathan. -----Original Message----- From: Holland, Richard Sent: Tuesday, 6 January 2004 4:27 p.m. To: Tony Cox; Warren, Jonathan Cc: ensembl-dev@ebi.ac.uk; das@biodas.org Subject: RE: DAS security Hi, Jonathan told me about this problem of restricting access to internal DAS sources by IP address in Ensembl, and I made a custom solution to it that we have been testing for about 2 months now, and appears to work just fine. Note that this is in addition to removing the code from Ensembl that allows users to add their own external DAS tracks - I'll leave that to Jonathan to explain. The solution works at the Ensembl level and not the DAS source level, but this is sufficient if you restrict the DAS server firewall to only allow connections to DAS from the web server that is running Ensembl. An explanation of how it works will follow after these instructions, but first there are a number of things that must be done to make it all work (ENSEMBL_HOME refers to the location of your Ensembl installation): 1. Think up some security groups for internal DAS sources to belong to. For instance, all human sources could belong to a group labelled 'human'. Sources can only belong to one group. 2. In ENSEMBL_HOME/conf/<>.ini (eg. Homo_sapiens.ini), define all your permanent internal DAS sources as normal in the DAS CONFIG section. Make sure you list all sources you want here regardless of groups, as users will not be able to add their own if you have implemented Jonathan's modifications as mentioned above. Now, for each DAS source, add a line reading: group = human ...or whatever group you want the DAS source to belong to. This option goes in with all the others, eg. linktext, linkURL, depth, strand etc. 3. Create a file called ENSEMBL_HOME/conf/Security.pm . Give it the same permissions as the other .pm scripts in ENSEMBL_HOME/conf. It should contain the following Perl code: ---BEGIN CODE--- #!/usr/local/bin/perl package Security; use strict; # Defines one function which takes a security group as a parameter, # and returns 1 if the current user is allowed to see that track, 0 if not. # The constructor sets the username. # --- Richard Holland, AgResearch, January 2004 # email richard at uwc dot net sub new { my $class = shift; my $user = shift; bless ({'_user'=>$user},$class); } sub checkSecurity { my $self = shift; my $securitygroup = shift; my $securityuser = $self->{'_user'}; # Allow access if no group restriction specified return 1 if not defined $securitygroup; # Block access if group restriction present and username not given return 0 if not defined $securityuser; # Use some kind of authentication here to look up the securitygroup value # against the securityuser value and determine if the user is allowed to see # this group. This is entirely customisable - this example just returns 1 to # say that permission is granted. The code should return 0 if permission is # denied. In our local installation, there is a call to a web service here using # SOAP::Lite which does the lookup for us. return 1; # Default to permission granted - do not leave this here! } 1; ---END CODE--- 4. Configure Apache to require authentication on the ENSEMBL_HOME/perl folder - the usual way to do this is by .htaccess and .htpasswd but it is up to you, as long as it requires authentication and can provide a named user for the CGI remote_user parameter. 5. Edit ENSEMBL_HOME/perl/contigview and find the line (right near the top) that reads: my $uca = new EnsEMBL::Web::UserConfigAdaptor( EnsWeb::species_defs->SITE_TYPE ); Insert the following code directly after it: ---BEGIN CODE--- # BEGIN ENSEMBL SECURITY PATCH use Security; my $remote_user = $cgi->remote_user; my $security = new Security($remote_user); my $dassources = EnsWeb::species_defs->ENSEMBL_INTERNAL_DAS_SOURCES; foreach (keys %$dassources) { if ($security->checkSecurity($dassources->{$_}->{'security'}) == 0) { warn "...access DENIED to DAS source $_ for ".($remote_user||''); delete $dassources->{$_}; } else { warn "...access granted to DAS source $_ for ".($remote_user||''); } } EnsWeb::species_defs->set_config($ENV{'ENSEMBL_SPECIES'},'ENSEMBL_INTERN AL_DAS_SOURCES',$dassources,1); # END ENSEMBL SECURITY PATCH ---END CODE--- 6. That's it! Once you have implemented some kind of authentication in the Security.pm module from step, you will find that access restriction works. Now, the explanation. By adding the group parameter to the internal DAS sources in the species configuration file, we can categorise DAS sources. By adding the Apache authentication to the perl folder, we can get a username. Hence, we can limit sources by username by looking at the group the source belongs to. The code assumes that if no group has been given in the config file, then the source can be viewed by anybody. If a group is given but the user is unauthenticated (if you get .htaccess wrong for example), then access is denied to that source. If both a group and a username are present, then the last part of Security.pm is reached, which performs the authentication lookup. In our local installation we have a database of group names and usernames which we look up via a quick web service query, but any mechanism could be used. The mechanism of the Security.pm routine is simple - return 1 if permission is granted, return 0 if not. The alteration to contigview simply waits for the DAS source configuration to be loaded, then loops through the internal DAS sources and gets their group names. It then passes the group name to the Security.pm module for checking. If permission is granted, no further action is taken. If it is not granted, then the external DAS source is completely deleted from the configuration hash, so that the rest of Ensembl never knew it even existed. Of course this happens only in the memory copy of the config - the config on disk is left untouched. You can see who is granted or denied by looking at the Ensembl logs - hence you can bugfix your allocation of groups and usernames. cheers, Richard --- Richard Holland Bioinformatics Database Developer ITS, Agresearch Invermay x3279 -----Original Message----- From: Tony Cox [mailto:avc@sanger.ac.uk] Sent: Tuesday, 23 September 2003 9:59 p.m. To: Warren, Jonathan Cc: ensembl-dev@ebi.ac.uk; Subject: Re: DAS security On Tue, 23 Sep 2003, Warren, Jonathan wrote: +>Hi +> +>We have ensembl and Dazzle installed locally and have internal IP +>issues where some groups are not supposed to be allowed to see certain +>tracks. There is no built in security model in Ensembl (although this is not to say Ensembl is built without regard to security!) Since it has always been an open source/data project we have not engineered a system for hiding some data form a subset of users. I think you would have to use either a proxy layer to filter data by IP address ranges or else you will have to embed some IP-based track switching in the drawing code. We'd be very interested to know if you come up with a nice solution! regards Tony +>Where is the best place to change the ensembl code to restrict access +>to only certain DSNs (if a change in the code is needed?). Is there +>one place that would stop people using the "Manage Sources" menu and +>would allow us to restrict the DAS sources currently configured in our +>HomoSapiens.ini file to certain users? +> +>Cheers +> +>Jonathan +> +> +> +>====================================================================== +>= +>Attention: The information contained in this message and/or attachments +>from AgResearch Limited is intended only for the persons or entities +>to which it is addressed and may contain confidential and/or privileged +>material. Any review, retransmission, dissemination or other use of, or +>taking of any action in reliance upon, this information by persons or +>entities other than the intended recipients is prohibited by AgResearch +>Limited. If you have received this message in error, please notify the +>sender immediately. +>====================================================================== = +> ****************************************************** Tony Cox Email:avc@sanger.ac.uk Sanger Institute WWW:www.sanger.ac.uk Wellcome Trust Genome Campus Head,Software Services Hinxton Tel: +44 1223 834244 Cambs. CB10 1SA Fax: +44 1223 494919 ****************************************************** ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. ======================================================================= ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================