-- Will Breaden Madden - 2015-02-21

CERN Git

This page contains information on the use of CERN Git repositories.

introduction

Git is better than Subversion.

CERN provides access-controlled Git repositories. In a nutshell, a repository is created usually for an individual project and its access permissions are controlled by e-groups. So, the access and editing permissions for users of the repository are inherited from an e-group. For a quick start, a recommended procedure is as follows:

  • Request a new e-group.
  • Become not only an owner of the e-group, but also a member.
  • Request a new repository (perhaps one for testing initially).
  • Wait for a time (anything from 20 minutes to 17 hours) for the e-group permissions to propagate to the repository.
  • Clone your new repository using Kerberos authentication.
  • Fuck shit up.
  • Push to the central repository.

links

The CERN central Git service is at https://git.cern.ch/. The CERN Information Technology Department page on the service is https://information-technology.web.cern.ch/services/git-service. The CERN Service Portal Knowledge Base articles for the Git service are here https://cern.service-now.com/service-portal/faq.do?se=git-service.

web representation of repositories

CERN repositories can be represented using Gitweb.

create new repository

To create a GIT repository, an e-group is needed for permissions.

create an e-group

The access and editing permissions associated with a CERN Git repository are inherited from a CERN e-group. Specifically, the members of the e-group are granted repository permissions -- not owners. For the development of a group project, an existing e-group can be used. For personal or experimental projects, a new e-group can be beneficial. Create an e-group at the following link:

Note that there is a naming convention for e-group names:

An example e-group name could be "atlas-sw-skunkworks". As of 2015-02-13, there is the start of a Git infrastructure e-group naming convention for user Git repositories:

atlas-sw-git-<AFS username>

For the purposes of CERN Git repository permissions, do not simply be the owner of the e-group; become a member of the e-group.

static e-groups and self-subscription policy

A static e-group means that the creator defines the policies for self-subscription and users are not granted access to the members list.

dynamic e-groups

Dynamic e-groups are defined by criteria such as experiment, department, status and location.

privacy policy

privacy policy comment
e-group Owner-Admins The owner and administrators only are granted access to the members list.
e-group members E-group members only are granted access to the members list.
CERN registered users CERN users only are granted access to the members list.
Open Any user is granted access to the members list.

owners and administrators

There is one owner and there can be multiple administrators of an e-group. Ownership can be transferred. Owners and administrators have the same usage rights of the e-group. When an e-group is deleted, the owner only can recreate a group with the same name 30 days afterwards.

posting restrictions

By default, owners and administrators only are granted the ability to post messages. This can be changed in the mail properties configuration.

archives

E-group archives are available via a URL of the following form:

http://cern.ch/group/<group name>

using e-groups for access control

e-groups can be used for granting access to Indico events. In order to do this, access the Indico event's management page and then access the protection section.

create a repository

To create a GIT repository, access CERN Forge (a portal for the management of SVN, Git and JIRA projects) and create a request for a GIT repository:

The following details are needed for the request:

  • the name of the repository to be created,
  • the e-group name and
  • the initial quota (optional).

The following procedure can be used to create a new repository:

The time for a new project on CERN Forge to change from status requested to status active on 2015-01-26 was ~7 minutes. The resulting CERN Forge page for the project could be something such as the following:

The resulting web listing for the project should be included here:

The link to the web representation of the project (CERN Gitweb) should be the following:

If the project is public, the resulting web listing for the project should be here:

The default access rights to a CERN Git repository are restricted to members of the e-group provided in the request.

CERN Git repository usage

clone repository

For the purposes of examples, a repository with the name "junk" is used. The Kerberos interface can be used to clone a Git repository using a command such as the following:

kinit ${USER}@CERN.CH
git clone https://:@git.cern.ch/kerberos/junk

To clone a Git repository from the LXPLUS cluster with LDAP authentication, the username should be specified in the URL:

git clone https://<username>@git.cern.ch/ldap/junk

For a public repository, a command like the following can be used:

git clone http://git.cern.ch/pub/junk

commits and pushes

For a new, empty repository, the push procedure should be initialised likely in the following way:

git push origin master

Following this, standard commits and pushes should be available.

README repository summary page

Gitweb can display a README file in the summary page of a CERN Git repository. This is used typically as a short introduction to the repository.

There are some CERN Knowledge Base articles on this:

There are two main steps involved in setting up README summary page functionality for a CERN Git repository. The first step is setting up a script that runs after pushing to the repository of git.cern.ch which copies a HTML README file such that it is displayed on Gitweb. The second step is setting the "gitweb.description" of the project in order to display appropriate information on the repository summary page RSS banner.

set up git-hook for README

Add a post-receive hook script using the following command:

git cat-file blob HEAD:README.html > $GIT_DIR/README.html

Then, when pushing to git.cern.ch, a file called "README.html" is searched for at the HEAD of the master branch of the repository and then put in the root of the directory ${GIT_DIR}. ${GIT_DIR} is a Git variable set by Git in the server that points to the root path of the repository. This is run with every push of the repository to git.cern.ch, so every time a new file README.html is pushed to the repository at git.cern.ch, Gitweb should display it.

Markdown is a good way to compose the README. A script to convert a Markdown README file to a HTML file compatible with CERN Git Gitweb (and a standaline HTML file perhaps for use with CERN SVN repositories) is available here:

This script depends on the program Pandoc. The program Markdown could also be used (though it does not feature LaTeX-style mathematics rendering). The Gitweb summary page for example project "junk" would be at the following URL:

In order to set up the post-receive script, clone the special CERN Git repository "git-hooks". Then, create the executable script "post-receive" in the directory of the repository "git-hooks" that corresponds to the name of the repository for which a README workflow is being prepared (e.g. "junk"). Ensure that the script (e.g. git-hooks/junk/post-receive) is executable. The content of the script should be as follows:

#!/bin/bash

git cat-file blob HEAD:README.html > $GIT_DIR/README.html

The HTML README file should not be a standalone HTML file because it is rendered as part of an existing web page for the repository. So, for example, the following HTML content would be appropriate:

<p>This text is normal.</p>
<p><b>This text is bold.</b></p>
<p><i>This text is italic.</i></p>
<p>This text is normal.</p>

Specifically, the content of the HTML README file is inserted into a DIV of the rendered Gitweb project summary page.

A procedure to do this is as follows:

kinit ${USER}@CERN.CH
git clone https://:@git.cern.ch/kerberos/git-hooks
cd git-hooks
touch junk/post-receive
cat >> junk/post-receive<<'EOF'
#!/bin/bash

git cat-file blob HEAD:README.html > ${GIT_DIR}/README.html
EOF
chmod 755 junk/post-receive
git add junk/post-receive
git commit -m 'add post-receive script for package junk'
git push

set up gitweb-description

By default, the "description" of a Git repository is an arbitrary string and the "owner" is the e-group that administers the repository. These variables are for the Gitweb summary page of a repository only. They do not change any other functionality of a repository. Note that to change the owner of a repository, a request must be submitted. In order to change these variables, clone the repository "gitolite-admin". Edit the configuration file of the repository under development in the repository gitolite-admin (e.g. gitolite-admin/conf/subs/junk.conf). For this configuration file, the following default variables are defined:

config gitweb.owner = "your egroup"
config gitweb.description = "use this variable to change description"

The full edited, resultant content of the gitolite-admin configuration file for the repository "junk" could be something such as the following:

repo junk
    #This is only for beautification.DOES NOT change any functionality
    config gitweb.owner = atlas-sw-skunkworks
    config gitweb.description = change gitweb.description variable in gitolite-admin
    RW+ = @atlas-sw-skunkworks

Changing, committing and then pushing to the CERN central Git repository updates the variables.

A procedure to do this is as follows:

kinit ${USER}@CERN.CH
git clone https://:@git.cern.ch/kerberos/gitolite-admin
cd gitolite-admin
cat > conf/subs/TTHbbLeptonic.conf<<'EOF'
repo TTHbbLeptonic
    config gitweb.owner = atlas-sw-skunkworks
    config gitweb.description = development and testing of ttH, H to bb prototype analysis package for l + jets, dilepton and boosted channels
    RW+ = @atlas-sw-skunkworks
EOF
git add conf/subs/TTHbbLeptonic.conf
git commit -m 'update Gitweb description'
git push

README workflow

A recommended workflow for generating the README in general development (i.e. following the git-hooks setup etc.) is as follows:

  • Edit the Markdown README file "README.md".
  • Compile the Markdown README file to a HTML file suitable for inclusion in a DIV of a CERN Gitweb summary page and to a standalone HTML file (perhaps suitable for CERN SVN repositories) using the script "compile_documentation.py" (noting that the program Pandoc must be available as a dependency of this script -- as must be the Python module docopt).

wget https://raw.githubusercontent.com/wdbm/CERN-projects/master/compile_documentation.py
python compile_documentation.py
rm compile_documentation.py

  • Git add the edited Markdown README file and the resultant README HTML files.

git add README.md
git add README.html
git add README_SVN.html

  • Commit and push.
Topic revision: r6 - 2015-02-21 - WilliamBreadenMadden
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback