[Biopython-dev] Updating http://biopython.open-bio.org/SRC/biopython/

[Peter]

On Tue, Nov 17, 2009 at 12:42 PM, Bartek Wilczynski
<bartek at rezolwenta.eu.org> wrote:
> On Tue, Nov 17, 2009 at 1:16 PM, Peter <biopython at maubp.freeserve.co.uk> wrote:
>> Dear all,
>>
>> Back when Biopython used CVS, we had an hourly checkout of
>> the code published here:
>> http://www.biopython.org/SRC/biopython/
>> http://biopython.open-bio.org/SRC/biopython/
>>
>> I did ask how this was setup (which username etc) but no one got
>> back to me. In any case, this can now be turned off - along with
>> making the Biopython CVS read only (OBF support call #857).
>>
>> I've managed to get this working again from the github repository,
>> although the route isn't ideal (and is all running under my username):
>
> Great!
>>
>> (1) At quarter past the hour, a cron job on dev.open-bio.org does
>> a "git pull" to update a local copy of the repository to that on github.
>> This doesn't need any passwords.
>>
>> (2) At 25mins past the hour, a cron job on www.biopython.org does
>> an rsync to get a copy of the repository from dev.open-bio.org (using
>> an SSH key for access).
>>
>> I tried having a single job running on dev.open-bio.org to push the
>> files to www.biopython.org but the host policies seem to block that.
>
> that's a pity. In case you haven't thought about it, I would only
> suggest to use some sort of a lockfile: In case github is very slow
> (happens every so often to me) it might occur that the second job will
> start before the first one is finished. Leading potentially to a
> broken repository for download. If the first job would create a
> lockfile before it starts and would delete it after it's done, the
> second job could condition the rsync operation on the existence of the
> file. This way, we could have delaysin syncing , rather than
> potentially broken repo. Of course, if we move to a simpler setup
> using only one host, this would not be needed.

Note we're not (yet) making a clone-able repository available on
www.biopython.org - this is just a simple hourly snapshot of the
source code (without the .git folder). A lock file might be possible
but seems overly complicated. I guess I could increase the time
delay if you are worried about github being slow sometimes.

>> It would be simpler to have everything done on www.biopython.org,
>> which would require git to be installed on that machine. This would
>> avoid any SSH security problems. Does that seem like a better idea?
>
> indeed, this would be best, and it only requires someone with root
> privileges to install a package.

That question was really aimed at the OBF admin team (hence my
CC'ing this to root-l), in case they have any better ideas.

> Also, it would be cool if the biopython user was somehow unlocked.
> (Currently no-one seems to have the password...)

I'm sure an OBF admin can reset it. Again, this is CC'd to root-l.

Peter