From hollandr at gis.a-star.edu.sg Wed Nov 10 04:50:13 2004 From: hollandr at gis.a-star.edu.sg (Richard HOLLAND) Date: Wed Nov 10 09:07:16 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema Message-ID: <6D9E9B9DF347EF4385F6271C64FB8D560126770C@BIONIC.biopolis.one-north.com> I have recently attempted to install BioSQL on Oracle using the most up-to-date version of the scripts in the CVS repository (downloaded three weeks ago, approx. 15th October), only to find that there were a number of bugs in them. My CVS access to pub.open-bio.org seems to have stopped working (although I can still ssh to it) so I am forced to email them to the list instead in the hope that someone else can make the changes for me...: * biosql-ora/BS-create-tablespaces.sql : - change @BS-defs to @BS-defs-local * biosql-ora/BS-create-schema-user.sql - change @BS-defs to @BS-defs-local * biosql-ora/BS-create-Biosql-usersyns.sql - add another 'set' line to prevent wrapping mid-word (and thus breaking) of SQL statements in the generated usersyns.sql script (the number is arbitrary, it works for me, and is better than the 80 char default): set lines 200 * biosql-ora/BS-defs.sql - add another definition allowing us to specify the instance name instead of relying on the environment variables to do this for us: define biosql_instance=biosql * biosql-ora/BS-create-all.sql - change all connect &sysdba/&dbapwd as sysdba to reference the instance variable like this: connect &sysdba@&biosql_instance/&dbapwd as sysdba - change all connect &biosql_owner/&biosql_pwd to reference the instance variable like this: connect &biosql_owner@&biosql_instance/&biosql_pwd * biosql-ora/BS-grants.sql - allow other users to see the primary keys, else they cannot insert or update even if they are in the admin/loader roles. Do this by removing a line from the last grant statement: AND object_name NOT LIKE '%_PK_SEQ' Also could someone add a note saying which privileges the BIOSQL_BASE_USER and BIOSQL_SCHEMA_CREATOR roles must have granted to them, because it's not that obvious at first. These ones work for me: @BS-defs-local create role &base_user; grant CREATE SESSION, CREATE SYNONYM, CREATE VIEW to &base_user; create role &schema_creator; grant CREATE PROCEDURE, CREATE ROLE, CREATE SEQUENCE, CREATE SESSION, CREATE SYNONYM, CREATE TRIGGER, CREATE TYPE, CREATE VIEW, CREATE TABLE to &schema_creator; It appears that the other bug I found, namely the definition for the view Term_Relationship_Term in the BS-create-Biosql-API2.sql referring to a non-existent column SG_Term_Assoc.Trm_Oid appears to have been fixed by someone on the 17th October by adding that column to the underlying table, so I won't post what I did about that one (I basically altered the view so that it pointed to obj_trm_oid instead, maybe not the best way to do things but never mind, it doesn't matter now...!). cheers, Richard PS. I am away now until Tuesday next week, if any questions need answering. I am aware that some changes were made on the 17th but I do not Richard Holland Bioinformatics Specialist Genome Institute of Singapore 60 Biopolis Street, #02-01 Genome, Singapore 138672 Tel: (65) 6478 8000 DID: (65) 6478 8199 Email: hollandr@gis.a-star.edu.sg --------------------------------------------- This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately. Please do not copy or use it for any purpose, or disclose its content to any other person. Thank you. --------------------------------------------- From hlapp at gnf.org Wed Nov 10 12:08:44 2004 From: hlapp at gnf.org (Hilmar Lapp) Date: Wed Nov 10 12:08:41 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema In-Reply-To: <6D9E9B9DF347EF4385F6271C64FB8D560126770C@BIONIC.biopolis.one-north.com> Message-ID: <2D55ABCF-333B-11D9-BC36-000A959EB4C4@gnf.org> Thanks for your suggestions, Richard. I'll include the schema instance, but without the literal '@' so that you may define it as empty too. The change from including BS-defs.sql to including BS-defs-local.sql is intentional in all scripts. You're supposed to copy BS-defs.sql, which serves as a template, to BS-defs-local.sql and edit it to suit your setup. As for the additional column and term_relationship_term view, Biojava needs this. Sorry for having been a day late with adding the column to the schema. I will also add all schema changes as DDL ALTER etc commands to the script sql/biosql-ora/migrate/1.0/migrate-all.sql. So, whenever you see a change in this script upon cvs update, just execute the added command and your schema is supposed to be up-to-date. -hilmar On Wednesday, November 10, 2004, at 01:50 AM, Richard HOLLAND wrote: > I have recently attempted to install BioSQL on Oracle using the most > up-to-date version of the scripts in the CVS repository (downloaded > three weeks ago, approx. 15th October), only to find that there were a > number of bugs in them. My CVS access to pub.open-bio.org seems to have > stopped working (although I can still ssh to it) so I am forced to > email > them to the list instead in the hope that someone else can make the > changes for me...: > > > > * biosql-ora/BS-create-tablespaces.sql : > > - change > > @BS-defs > > to > > @BS-defs-local > > * biosql-ora/BS-create-schema-user.sql > > - change > > @BS-defs > > to > > @BS-defs-local > > * biosql-ora/BS-create-Biosql-usersyns.sql > > - add another 'set' line to prevent wrapping > mid-word (and thus breaking) of SQL statements in the generated > usersyns.sql script (the number is arbitrary, it works for me, and is > better than the 80 char default): > > set lines 200 > > * biosql-ora/BS-defs.sql > > - add another definition allowing us to specify > the instance name instead of relying on the environment variables to do > this for us: > > define biosql_instance=biosql > > * biosql-ora/BS-create-all.sql > > - change all > > connect &sysdba/&dbapwd as sysdba > > to reference the instance variable like this: > > connect &sysdba@&biosql_instance/&dbapwd as sysdba > > - change all > > connect &biosql_owner/&biosql_pwd > > to reference the instance variable like this: > > connect > &biosql_owner@&biosql_instance/&biosql_pwd > > * biosql-ora/BS-grants.sql > > - allow other users to see the primary keys, else > they cannot insert or update even if they are in the admin/loader > roles. > Do this by removing a line from the last grant statement: > > AND object_name NOT LIKE '%_PK_SEQ' > > > > Also could someone add a note saying which privileges the > BIOSQL_BASE_USER and BIOSQL_SCHEMA_CREATOR roles must have granted to > them, because it's not that obvious at first. These ones work for me: > > > > @BS-defs-local > > create role &base_user; > > grant > > CREATE SESSION, > > CREATE SYNONYM, > > CREATE VIEW > > to &base_user; > > create role &schema_creator; > > grant > > CREATE PROCEDURE, > > CREATE ROLE, > > CREATE SEQUENCE, > > CREATE SESSION, > > CREATE SYNONYM, > > CREATE TRIGGER, > > CREATE TYPE, > > CREATE VIEW, > > CREATE TABLE > > to &schema_creator; > > > > It appears that the other bug I found, namely the definition for the > view Term_Relationship_Term in the BS-create-Biosql-API2.sql referring > to a non-existent column SG_Term_Assoc.Trm_Oid appears to have been > fixed by someone on the 17th October by adding that column to the > underlying table, so I won't post what I did about that one (I > basically > altered the view so that it pointed to obj_trm_oid instead, maybe not > the best way to do things but never mind, it doesn't matter now...!). > > > > cheers, > > Richard > > > > PS. I am away now until Tuesday next week, if any questions need > answering. I am aware that some changes were made on the 17th but I do > not > > > > Richard Holland > > Bioinformatics Specialist > > Genome Institute of Singapore > > 60 Biopolis Street, #02-01 Genome, Singapore 138672 > > Tel: (65) 6478 8000 DID: (65) 6478 8199 > > Email: hollandr@gis.a-star.edu.sg > > > > --------------------------------------------- > > This email is confidential and may be privileged. If you are not the > intended recipient, please delete it and notify us immediately. Please > do not copy or use it for any purpose, or disclose its content to any > other person. Thank you. > > --------------------------------------------- > > > > _______________________________________________ > BioSQL-l mailing list > BioSQL-l@open-bio.org > http://open-bio.org/mailman/listinfo/biosql-l > -- ------------------------------------------------------------- Hilmar Lapp email: lapp at gnf.org GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 ------------------------------------------------------------- From hollandr at gis.a-star.edu.sg Mon Nov 15 20:08:16 2004 From: hollandr at gis.a-star.edu.sg (Richard HOLLAND) Date: Mon Nov 15 20:07:09 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema Message-ID: <6D9E9B9DF347EF4385F6271C64FB8D56012677C2@BIONIC.biopolis.one-north.com> Thanks Hilmar. I already understood the BS-defs/BS-defs-local situation, that was why I was surprised to find two scripts still referring to BS-defs to load constants when all the others were referring to BS-defs-local instead. Mark found one other potential problem - we're not sure if this should be fixed or not. When loading Genbank records, we hit a record with a very large open reading frame translation stored as one peptide, which was too big to fit in the appropriate column. The column failing was bioentry_qualifer_value.value, which when expanded to 300 characters in length was able to store it. Not sure if this should be changed in BioSQL or not? Richard Holland Bioinformatics Specialist GIS extension 8199 --------------------------------------------- This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately. Please do not copy or use it for any purpose, or disclose its content to any other person. Thank you. --------------------------------------------- > -----Original Message----- > From: Hilmar Lapp [mailto:hlapp@gnf.org] > Sent: Thursday, November 11, 2004 1:09 AM > To: Richard HOLLAND > Cc: biosql-l@open-bio.org > Subject: Re: [BioSQL-l] Updates to biosql Oracle schema > > Thanks for your suggestions, Richard. I'll include the schema instance, > but without the literal '@' so that you may define it as empty too. > > The change from including BS-defs.sql to including BS-defs-local.sql is > intentional in all scripts. You're supposed to copy BS-defs.sql, which > serves as a template, to BS-defs-local.sql and edit it to suit your > setup. > > As for the additional column and term_relationship_term view, Biojava > needs this. Sorry for having been a day late with adding the column to > the schema. I will also add all schema changes as DDL ALTER etc > commands to the script sql/biosql-ora/migrate/1.0/migrate-all.sql. So, > whenever you see a change in this script upon cvs update, just execute > the added command and your schema is supposed to be up-to-date. > > -hilmar > > On Wednesday, November 10, 2004, at 01:50 AM, Richard HOLLAND wrote: > > > I have recently attempted to install BioSQL on Oracle using the most > > up-to-date version of the scripts in the CVS repository (downloaded > > three weeks ago, approx. 15th October), only to find that there were a > > number of bugs in them. My CVS access to pub.open-bio.org seems to have > > stopped working (although I can still ssh to it) so I am forced to > > email > > them to the list instead in the hope that someone else can make the > > changes for me...: > > > > > > > > * biosql-ora/BS-create-tablespaces.sql : > > > > - change > > > > @BS-defs > > > > to > > > > @BS-defs-local > > > > * biosql-ora/BS-create-schema-user.sql > > > > - change > > > > @BS-defs > > > > to > > > > @BS-defs-local > > > > * biosql-ora/BS-create-Biosql-usersyns.sql > > > > - add another 'set' line to prevent wrapping > > mid-word (and thus breaking) of SQL statements in the generated > > usersyns.sql script (the number is arbitrary, it works for me, and is > > better than the 80 char default): > > > > set lines 200 > > > > * biosql-ora/BS-defs.sql > > > > - add another definition allowing us to specify > > the instance name instead of relying on the environment variables to do > > this for us: > > > > define biosql_instance=biosql > > > > * biosql-ora/BS-create-all.sql > > > > - change all > > > > connect &sysdba/&dbapwd as sysdba > > > > to reference the instance variable like this: > > > > connect &sysdba@&biosql_instance/&dbapwd as sysdba > > > > - change all > > > > connect &biosql_owner/&biosql_pwd > > > > to reference the instance variable like this: > > > > connect > > &biosql_owner@&biosql_instance/&biosql_pwd > > > > * biosql-ora/BS-grants.sql > > > > - allow other users to see the primary keys, else > > they cannot insert or update even if they are in the admin/loader > > roles. > > Do this by removing a line from the last grant statement: > > > > AND object_name NOT LIKE '%_PK_SEQ' > > > > > > > > Also could someone add a note saying which privileges the > > BIOSQL_BASE_USER and BIOSQL_SCHEMA_CREATOR roles must have granted to > > them, because it's not that obvious at first. These ones work for me: > > > > > > > > @BS-defs-local > > > > create role &base_user; > > > > grant > > > > CREATE SESSION, > > > > CREATE SYNONYM, > > > > CREATE VIEW > > > > to &base_user; > > > > create role &schema_creator; > > > > grant > > > > CREATE PROCEDURE, > > > > CREATE ROLE, > > > > CREATE SEQUENCE, > > > > CREATE SESSION, > > > > CREATE SYNONYM, > > > > CREATE TRIGGER, > > > > CREATE TYPE, > > > > CREATE VIEW, > > > > CREATE TABLE > > > > to &schema_creator; > > > > > > > > It appears that the other bug I found, namely the definition for the > > view Term_Relationship_Term in the BS-create-Biosql-API2.sql referring > > to a non-existent column SG_Term_Assoc.Trm_Oid appears to have been > > fixed by someone on the 17th October by adding that column to the > > underlying table, so I won't post what I did about that one (I > > basically > > altered the view so that it pointed to obj_trm_oid instead, maybe not > > the best way to do things but never mind, it doesn't matter now...!). > > > > > > > > cheers, > > > > Richard > > > > > > > > PS. I am away now until Tuesday next week, if any questions need > > answering. I am aware that some changes were made on the 17th but I do > > not > > > > > > > > Richard Holland > > > > Bioinformatics Specialist > > > > Genome Institute of Singapore > > > > 60 Biopolis Street, #02-01 Genome, Singapore 138672 > > > > Tel: (65) 6478 8000 DID: (65) 6478 8199 > > > > Email: hollandr@gis.a-star.edu.sg > > > > > > > > --------------------------------------------- > > > > This email is confidential and may be privileged. If you are not the > > intended recipient, please delete it and notify us immediately. Please > > do not copy or use it for any purpose, or disclose its content to any > > other person. Thank you. > > > > --------------------------------------------- > > > > > > > > _______________________________________________ > > BioSQL-l mailing list > > BioSQL-l@open-bio.org > > http://open-bio.org/mailman/listinfo/biosql-l > > > -- > ------------------------------------------------------------- > Hilmar Lapp email: lapp at gnf.org > GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 > ------------------------------------------------------------- > From hlapp at gnf.org Mon Nov 15 20:24:40 2004 From: hlapp at gnf.org (Hilmar Lapp) Date: Mon Nov 15 20:22:46 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema In-Reply-To: <6D9E9B9DF347EF4385F6271C64FB8D56012677C2@BIONIC.biopolis.one-north.com> References: <6D9E9B9DF347EF4385F6271C64FB8D56012677C2@BIONIC.biopolis.one-north.com> Message-ID: <4953DE82-376E-11D9-9684-000A95AE92B0@gnf.org> On Nov 15, 2004, at 5:08 PM, Richard HOLLAND wrote: > The column failing was bioentry_qualifer_value.value, which when > expanded to 300 characters in > length was able to store it. Not sure if this should be changed in > BioSQL or not? I assume you mean 3000, not 300? In the DDL script the width is actually expanded to 4000, the full size. Also, if it's a translation then it will have been in the feature table, and the respective table would then be seqfeature_qualifier_value. Apparently both column width expansions are in the DDL, but not in the migration script. I added them both to the migration script too (sql/biosql-ora/migrate/1.0/migrate-all.sql). -hilmar -- ------------------------------------------------------------- Hilmar Lapp email: lapp at gnf.org GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 ------------------------------------------------------------- From hollandr at gis.a-star.edu.sg Mon Nov 15 20:26:15 2004 From: hollandr at gis.a-star.edu.sg (Richard HOLLAND) Date: Mon Nov 15 20:24:57 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema Message-ID: <6D9E9B9DF347EF4385F6271C64FB8D56012677CD@BIONIC.biopolis.one-north.com> Yes, 3000 is what we used. I think that particular value was obtained from Len's schema - we have been using both yours and Len's in parallel to see which works best for us. Richard Holland Bioinformatics Specialist GIS extension 8199 --------------------------------------------- This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately. Please do not copy or use it for any purpose, or disclose its content to any other person. Thank you. --------------------------------------------- > -----Original Message----- > From: Hilmar Lapp [mailto:hlapp@gnf.org] > Sent: Tuesday, November 16, 2004 9:25 AM > To: Richard HOLLAND > Cc: mark.schreiber@group.novartis.com; biosql-l@open-bio.org > Subject: Re: [BioSQL-l] Updates to biosql Oracle schema > > > On Nov 15, 2004, at 5:08 PM, Richard HOLLAND wrote: > > > The column failing was bioentry_qualifer_value.value, which when > > expanded to 300 characters in > > length was able to store it. Not sure if this should be changed in > > BioSQL or not? > > I assume you mean 3000, not 300? In the DDL script the width is > actually expanded to 4000, the full size. Also, if it's a translation > then it will have been in the feature table, and the respective table > would then be seqfeature_qualifier_value. > > Apparently both column width expansions are in the DDL, but not in the > migration script. I added them both to the migration script too > (sql/biosql-ora/migrate/1.0/migrate-all.sql). > > -hilmar > -- > ------------------------------------------------------------- > Hilmar Lapp email: lapp at gnf.org > GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 > ------------------------------------------------------------- From hlapp at gnf.org Mon Nov 15 21:59:09 2004 From: hlapp at gnf.org (Hilmar Lapp) Date: Mon Nov 15 21:57:14 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema In-Reply-To: <6D9E9B9DF347EF4385F6271C64FB8D56012677CD@BIONIC.biopolis.one-north.com> References: <6D9E9B9DF347EF4385F6271C64FB8D56012677CD@BIONIC.biopolis.one-north.com> Message-ID: <7C8BFC87-377B-11D9-9684-000A95AE92B0@gnf.org> Len's Hsql schema is in the repository too BTW. Aside from that, please do feed problems back to the mailing list or the whole idea behind biosql becomes obsolete. -hilmar On Nov 15, 2004, at 5:26 PM, Richard HOLLAND wrote: > Yes, 3000 is what we used. I think that particular value was obtained > from Len's schema - we have been using both yours and Len's in parallel > to see which works best for us. > > Richard Holland > Bioinformatics Specialist > GIS extension 8199 > > --------------------------------------------- > This email is confidential and may be privileged. If you are not the > intended recipient, please delete it and notify us immediately. Please > do not copy or use it for any purpose, or disclose its content to any > other person. Thank you. > --------------------------------------------- > >> -----Original Message----- >> From: Hilmar Lapp [mailto:hlapp@gnf.org] >> Sent: Tuesday, November 16, 2004 9:25 AM >> To: Richard HOLLAND >> Cc: mark.schreiber@group.novartis.com; biosql-l@open-bio.org >> Subject: Re: [BioSQL-l] Updates to biosql Oracle schema >> >> >> On Nov 15, 2004, at 5:08 PM, Richard HOLLAND wrote: >> >>> The column failing was bioentry_qualifer_value.value, which when >>> expanded to 300 characters in >>> length was able to store it. Not sure if this should be changed in >>> BioSQL or not? >> >> I assume you mean 3000, not 300? In the DDL script the width is >> actually expanded to 4000, the full size. Also, if it's a translation >> then it will have been in the feature table, and the respective table >> would then be seqfeature_qualifier_value. >> >> Apparently both column width expansions are in the DDL, but not in the >> migration script. I added them both to the migration script too >> (sql/biosql-ora/migrate/1.0/migrate-all.sql). >> >> -hilmar >> -- >> ------------------------------------------------------------- >> Hilmar Lapp email: lapp at gnf.org >> GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 >> ------------------------------------------------------------- > > -- ------------------------------------------------------------- Hilmar Lapp email: lapp at gnf.org GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 ------------------------------------------------------------- From mark.schreiber at group.novartis.com Sun Nov 21 20:16:07 2004 From: mark.schreiber at group.novartis.com (mark.schreiber@group.novartis.com) Date: Sun Nov 21 20:14:13 2004 Subject: [BioSQL-l] Updates to biosql Oracle schema Message-ID: Actually this was my mistake. The thing I changed (in Lens Schema) was: ALTER TABLE "BIOJAVA_LEN"."BIOENTRY_QUALIFIER_VALUE" MODIFY ("VALUE" VARCHAR2(300)) Some of those viruses have bloody long descriptions! - Mark Mark Schreiber Principal Scientist (Bioinformatics) Novartis Institute for Tropical Diseases (NITD) 10 Biopolis Road #05-01 Chromos Singapore 138670 www.nitd.novartis.com phone +65 6722 2973 fax +65 6722 2910 Hilmar Lapp 11/16/2004 09:24 AM To: "Richard HOLLAND" cc: Mark Schreiber/GP/Novartis@PH, Subject: Re: [BioSQL-l] Updates to biosql Oracle schema On Nov 15, 2004, at 5:08 PM, Richard HOLLAND wrote: > The column failing was bioentry_qualifer_value.value, which when > expanded to 300 characters in > length was able to store it. Not sure if this should be changed in > BioSQL or not? I assume you mean 3000, not 300? In the DDL script the width is actually expanded to 4000, the full size. Also, if it's a translation then it will have been in the feature table, and the respective table would then be seqfeature_qualifier_value. Apparently both column width expansions are in the DDL, but not in the migration script. I added them both to the migration script too (sql/biosql-ora/migrate/1.0/migrate-all.sql). -hilmar -- ------------------------------------------------------------- Hilmar Lapp email: lapp at gnf.org GNF, San Diego, Ca. 92121 phone: +1-858-812-1757 -------------------------------------------------------------