You are here: GSI Wiki>Linux Web>GitUsage (2021-11-26, ThomasRoth)Edit Attach


Introduction

Git is a version control system (VCS) similar to Subversion and CVS. The major difference compared to SubVersion (and the like) is its distributed approach. This means that no central repository server is necessary. Everyone has private repositories, and collaboration is enabled by synchronization of individual repositories.

http://git-scm.com/

Host your git repositories at:

http://github.com/
or
http://gitlab.com/ or
https://git.gsi.de

Customize

Task

You want to create your private Git configuration.

Solution

Personal settings -- user name and mail address.
> git config --global user.name "Your Name Comes Here"
> git config --global user.email you@yourdomain.example.com

Show current configuration.

> git config --list

Discussion

User specific configuration is stored in a file called '~/.gitconfig'. Git reads and writes to this file specifically by passing the '--global' option. Repository configuration is located in the Git directory '.git/config'.

Other available options:
> git config --global color.diff auto
> git config --global color.status auto
> git config --global color.branch auto
> git config --global core.editor vim
> git config --global merge.tool vimdiff
> git help config

Enable the colorization of the command-line outputs, set your default editor and your preferred diff tool.

> git config --global alias.co checkout
> git config --global alias.br branch
> git config --global alias.ci commit
> git config --global alias.st status
> git config --global alias.unstage 'reset HEAD --'
> git config --global alias.last 'log -1 HEAD'

If you don't want to type the entire text of each of the Git commands, you can easily set up an alias for each command (see examples above).

Initialize New Repositories

Task

You want to put text, code, what ever under version control.

Solution To create a new repository, use git init. Before doing so you need to cd into the directory storing the files you want to put under version control.
> cd path/to/your/files
> git init
> git add .
> git commit -m 'initial project version'

git init creates the repository in ".git/". With the add command you will add all files in the local directory into the index of files tracked by Git. Finally you commit them into the repository.

Discussion

Basically, files in git can have three states:

  • "Committed" means that the data is safely stored in your local repository (Git directory).
  • "Modified" means that you have changed the file in your working directory but have not committed it to your repository.
  • "Staged" means that you have marked a modified file in its current version to go into your next commit to the repository

The Git directory '.git/' is where Git stores the meta-data and object database for your files. This is the most important part of Git, and it is what is copied when you clone a repository from another computer. The working directory is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

Cloning Existing Repositories

Task

You want to clone a remote repository (from another computer) to your the local machine.

Solution

Clone via SSH, the Git protocol or, HTTP:
> git clone [url]
> git remote add [shortname] [url]
> git remote -v

You can add more remote origins with add by defining an associated short-name. List all remotes for a particular repository with the -v option.

Discussion

The clone command creates a new directory named like the project, containing the working tree and a special top-level directory named '.git/' which contains all the information about the history of the project. Afterwards you will have a checkout copy of the master. The clone command automatically sets up your local master branch to track the remote master branch on the server you cloned from (assuming the remote has a master branch).
> git clone ~/path/name.git 
> git clone ssh://user@host/~/path/name.git
> git clone git://host/path/name.git
> git remote add test git://host/path/name.git 
> git remote -v 

Add a new Git repository using a short-name so you can reference it easily. To see which remote servers you have configured, you can run the remote command. You can also specify '-v', which shows you the URL that Git has stored for the short-name.

Cloning at GSI

To make git even harder to use, there are several options here. In general the advice is to use ssh instead of git due to the mechanisms used on gitorious.gsi.de.
  • git clone ssh://git@gitorious.gsi.de/site-cookbooks/pacemaker.git pacemaker
  • git clone git://gitorious.gsi.de/site-cookbooks/pacemaker.git pacemaker
Using the second form with git: might also cause future problems with git push, as indicated above.

It is possible that git clone alone is not sufficient: the cloned repository has to be informed about the remote location where it has just been cloned from (no words):
  • git remote add pacemaker git://gitorious.gsi.de/site-cookbooks/pacemaker.git
In other words, git clone is just a preparation for the checkout. (And the automatic setting of local master to remote master seems to be very unstable.)

Cloning with branches

There is always a possibility to make an overly sophisticated process even harder to use.
  • The obvious answer to a change such as the release of a new Debian version is to create a repository with the given name, as in Jessie-Fai or something similar.
  • Instead, you should create a branch inside the existing repository and give the name to it. This gives you the dubious pleasure of having to carry all the old history of the old and deprecated original repository, but it makes usage somewhat more complicated, so that must be your choice.

  • git clone ssh://git@gitorious.gsi.de/Repo/Repo.git -b Branch checks it out.

Cloning and Checking out remote repositories

As stated above, git clone does not do the obvious: in addition to clone, you have to checkout, too. But what if there is no master branch on the remote? By default, clone will set up you local copy to track the remote master only. Without it, there is nothing to checkout

Solution

Check the state of the remote with branches
> git branches -r
>   origin/production ①
>  origin/refactored
>  origin/test

① Example

In this case, check out with git checkout:
> git checkout -b test origin/test
> ls test

Hopefully this has finally written the desired files to the working directory.

Cloning all the site-cookbooks

Use the following script, e.g. as get-repos.sh, to get the full list of repositories to clone or pull from:
#!/bin/bash

COOKIES="./.cookies"
URL="gitorious.gsi.de"

read -s -p "Enter $URL password for '$1': " password
curl --insecure --silent --data "email=$1" --data "password=$password" --cookie-jar $COOKIES https://$URL/sessions > /dev/null
curl --insecure --silent --cookie $COOKIES http://$URL/$2.xml | xmlstarlet select -t -v '/project/repositories/mainlines/repository/clone_url' | sed 's/:\/\//@/g' | sed 's|de/site|de:/site|g'
rm $COOKIES

The parameters are your email and the remote directory of repositories, so:

get-repos.sh t.roth@gsi.de site-cookbooks

Put the output into the file repos and run

#!/usr/bin/env ruby

@infile= ARGV[0]
datei =File.readlines(@infile)

datei.each do |s| 
  name=s.split('/')[2].split(".")[0].chomp
  puts `git clone #{s.chomp} #{name}`
end

Synchronize with Remotes

Task

You want to publish changes made to your repository upstream to make them publicly available or your want to keep track of changes in the remote repository and merge them with your work.

Solution

> git fetch origin
> git pull
> git push origin master

fetch goes out to the remote project and pulls down all the data from that remote project that you do not have yet. Afterwards, you should have references to all the branches of the remote, which you can merge in or inspect at any time. It does not automatically merge it with any of your work or modify what you’ are currently working on. You have to merge it manually into your work.

Use the pull command to automatically fetch and then merge a remote branch into your current branch (if possible). Upload all changes upstream to a remote repository with push. If you want to push your master branch to your origin server (again, cloning generally sets up both of those names for you automatically), then you can run push without parameters. If you and someone else clones at the same time and they push upstream and then you push upstream, your push will rightly be rejected. You’ll have to pull down their work first and incorporate it into yours before you wi’ll be allowed to push.

Problem

git push does not work, although the steps above have been carried out before. The error message might read

fatal: protocol error: expected sha/ref, got '
----------------------------------------------
The git:// url is read-only. Please see http://gitorious.gsi.de/site-cookbooks/COOKBOOK for the push url, if you're a committer.
----------------------------------------------'

Try

Check the remotes of the current repository
> git remote

If this only says gitorious, only the http-url is known. To push, one needs the ssh-url instead.

Solution

> git remote add gitorious-ssh 'git@gitorious.gsi.de:fai/fai_config.git' ①
> git push gitorious-ssh  ②

① Add gitorious-ssh (fai_config is an example!!) as one of the remotes. ② Push explicitly to gitorious-ssh.

It might also help to try another command:
  • git remote set-url --push origin git@gitorious.gsi.de:/site-cookbooks/COOKBOOK.git
  • git push origin master
After that, even git push alone might work.

Create a Remote Repository

Task

You want to develop code on your local box, but the source control management should be placed on a remote host.

Solution

Create a bare repository and add it as origin.
> ssh user@host 'git init --bare [path]'
> git remote add [short-name] [url] 
> git push --all

Discussion

① Create a bare remote repository. ② Connect this empty repository as origin to the local repository. Work locally as usually, add files and commit them. ③ Push the entire local repository to its new remote origin.

> ssh user@host 'git init --bare ~/path/name.git' ①
> cd /path/name
> git init
> git remote add origin ssh://user@host/~/path/name.git ②
> git add .; git commit -m "Initial commit"
> git push --all ③ 

Clone bare repositories from existing ones and copy them to the remote server.

> git clone --bare . ./name.git
> scp -r name.git/ user@host:path/to/repos
> git remote add origin ssh://user@host/~/path/name.git
> git config branch.master.remote origin ④

④ You can use config to assign a default remote to a given branch. This default remote will be used to push that branch unless otherwise specified. This is already done for you when you use git clone, allowing you to use git push without any arguments to push the local master branch to update the origin repository's master branch.

Create a Remote Repository at GSI

The repository created in the steps above quite uselessly resides on your local disk. Of course you want to put your stuff on a server. In our case this is gitorious.gsi.de.
  • Most often the steps above do not seem to work.
  • The safest way seems to be to login to gitorious via the website http://gitorious.gsi.de and create a repository via the given menus.
    • The overview page of the newly created repo (or the existing ones) shows a box for each repo with Clone & push urls. There is a question mark in that box which leads to useful instructions on how to deal with the remote repo.
    • This boils down to e.g. git remote add origin git@gitorious.gsi.de:/site-cookbooks/COOKBOOK.git and git push gitorious master

*As of 2015-04-02*, git remote add origin git@gitorious.gsi.de:robinhood/robinhood.git and git push gitorious master results in an error.
Instead, the following commands really copy the desired files and directories to Gitorious:
git remote add gitorious-ssh 'git@gitorious.gsi.de:robinhood/robinhood.git'
git push gitorious-ssh master

Version Control

Add and Remove Files

Task

You want to manipulate the list of files tracked by the version control.

Solution

Git maintains a snapshot of the tree's contents in a special staging area called index. At the beginning, the content of the index will be identical to that of the head. To update the index with the new contents use add. Once part of the index you can move/remove files or directories with mv/rm.
> git add .
> git add path/to/file
> git rm path/to/file
> git mv path/to/file path/to/destination

Discussion

For a files present in the index use ls-files.
> git ls-files

If you want to ignore files you can create file listing pattern that match the names in '.gitignore' for each repository.
# a comment - this is ignored
*.a       # no .a files
!lib.a    # but do track lib.a, even though you're ignoring .a files above
/TODO     # only ignore the root TODO file, not subdir/TODO
build/    # ignore all files in the build/ directory
doc/*.txt # ignore doc/notes.txt, but not doc/server/arch.txt

The rules for the patterns you can put in the .gitignore file are as follows: Blank lines or lines starting with # are ignored. Standard glob patterns work. You can end patterns with a forward slash (/) to specify a directory. You can negate a pattern by starting it with an exclamation point (!).

Personal ignore patterns for all repositories may be defined in a central file, e.g. '~/.gitignore', using the same simple syntax. You have to configure git to notice this file:
> git config --global core.excludesfile my_global_gitignore

Commit Changes

Task

You want to put local modifications of your source into the version control.

Solution

Once you have created / edited some files in there, you can check what you have changed by
  • git status
When ready, commit changes known by the index. To make life harder, changed files have to be added before commit, even if they had been gitted before. At least with option -a, git commit will update the index with any files that you've modified or removed and create a commit, all in one step.
> git commit 
> git commit -a
> git commit -a -m 'descripton'

Unless using the -m 'description' option to add the commit message, your standard editor will be opened for you to enter this.

Inspecting the Version History

Task

You want to take a close look at changes recorded by the version control.

Solution

Every change in the history of a project is represented by a commit. It reflects the physical state of a tree with a description of how we got there and why. It consists of SHA1 values for the commit, tree and parent, as well as the author and committer name with time-stamps. Comment and a diff to the parent are indented.
> git show
> git log

Every commit has a 40-hex-digit ID, it is a globally unique name for this commit. Following the chain of parents eventually will take you back to the beginning of the project.

Discussion

With log you can display a summary of past commits with author and commit data, as well as the commit message. If you want to see some abbreviated stats for each commit, you can use the --stat option.
> git log --stat
> git log --pretty=oneline
> git log --pretty=format:"%h - %an, %ar : %s"
> git log --pretty=format:"%h %s" --graph
> git log --since=2.weeks

Inspect Differences

Task

You want to examine changes between specific versions in detail.

Solution

For keeping track of what you're about to commit use diff: shows the difference between the index and your working tree (changes that would not be included if you ran commit). The option --cached shows differences between head and the index (that would be committed if you ran commit). To see the difference between head and working tree (what would be committed if you ran commit -a), use diff HEAD. A brief per-file summary of the above is given by status.
> git diff
> git diff --cached
> git diff HEAD
> git status

Manage Variations of Code-lines by Branching

Task

You want to develop new features, try out new ideas or change something where you cannot estimate the consequences. Therefore you need to develop on a different, independent version of your code base.

Solution

A single Git repository can track development on multiple branches. It does this by keeping a list of heads which reference the latest commit on each branch. List all branches (asterisk marks the currently checked-out branch.) with branch:
> git branch

Create a new branch referencing the same point in history as the current branch by appending a name, delete a branch with option '-d'. Update the working directory to reflect the version referenced with checkout:
> git branch  
> git branch -d 
> git checkout 

Git allows lines of development to diverge and then reconverge, and the point where two lines of development reconverge is called a merge.
> git merge <branch>

Create a Snapshot of a Repository

Task

You want a name or label referring to an important snapshot in time, consistent across your repository.

Solution

tag
> git tag
> git tag -a [tag-version] -m [comment]
> git show [tag-version]
> git push origin [tag-verison]

Discussion

① Search for tags with a particular pattern. ② Create a tag by specifying -a followed by the version number. The -m specifies a tagging message which is stored with the tag.

Another way to tag commits is with a lightweight tag. This is basically the commit checksum stored in a file. No other information is kept. To create a lightweight tag, don't supply the -a, -s, or -m option. You can also tag commits after you have moved past them, using the commit checksum.
> git tag -l 'v1.*' ①
> git tag -a v1.4 -m 'my version 1.4' ②
> git tag v1.4-p12
> git tag -a v1.2 9fceb02
> git show v1.4 ③
> git push origin v1.4  ④
> git push origin --tags

③ You can see the tag data along with the commit that was tagged by using the show command. ④ By default, push does not transfer tags to remote servers. You will have to explicitly push tags to a shared server after you have created them. If you have a lot of tags that you want to push up at once, you can also use the '--tags' option.

Submodules

Task

You want to have Git repositories as sub-directories of your current working repository.

Solution

Submodules allow you to keep a Git repository as a sub-directory of another Git repository. This lets you clone another repository into your project and keep your commits separate. ① Clone the external repository into your sub-directory. First you notice the '.gitmodules' file. This is a configuration file that stores the mapping between the project's URL and the local sub-directory you have pulled it into.
> git submodule add git://host/path/name.git name/ ①
> git submodule init ②
> git submodule update

② When cloning a project with submodules, you must run two commands: submodule init to initialize your local configuration file, and submodule update to fetch all the data from that project and check out the appropriate commit listed in your super-project

Undoing Things

Task

When stuff goes wrong, you would need to reverse file operations or undo commits.

Solution

① Redo a commit with --amend to add missing files or to change the commit message.
> git commit --amend ①
> git reset HEAD file.cpp ② 
> git checkout -- file.cpp ③

② Remove a file from the staging area with reset. ③ Discard the changes you have made in you working directory by checkout.

Working with git

If you are very lucky, after running all the steps above you have a working system of remote repositories and local copies.

In that improbable case, the workflow would be
  • cd your/local/files
  • git pull
  • Work on your stuff
  • Check with git status
  • git commit -a -m 'description'
  • git push

gitci script

Something like the following Rudi script (gitci) might get even closer to svn ci

#!/usr/bin/env ruby
require 'getoptlong'

input='Upload'
opts = GetoptLong.new(
                      [ '--message', '-m', GetoptLong::REQUIRED_ARGUMENT  ]
)
opts.each do |opt,arg|
  case opt
  when '--message' then input=arg.to_s
  end
end

puts `git commit -a -m "#{input}"`
puts `git push` 

Multiple remotes

A local git repository may have mutliple remotes:

$ git remote -v
origin      git@git.example.com:myself/project.git (fetch)
origin      git@git.example.com:myself/project.git (push)
upstream    https://git.example.com/group/project.git (fetch)
upstream    https://git.example.com/group/project.git (push)
otherfork   https://git.example.com/smartace/project.git (fetch)
otherfork   https://git.example.com/smartace/project.git (push)

=> 2 Remotes.

  • An welches Remote ich pushe, ist pro lokalem Branch in meinem git Repo konfiguriert oder undefiniert.

Look at the local branches:
$ git branch
  feature-xyz
* feature-xyz-take2
  issue-392
  issue-392-fixed
  master

= 5 Branches.

  • Um herauszufinden, wohin bestimmte Branches gepusht werden sollen, schau ich mir die git config an. Hierzu hab ich 2 Moeglichkeiten:
$ cat .git/config
[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    fetch = +refs/heads/*:refs/remotes/origin/*
    url = https://github.com/volatilityfoundation/volatility.git
[branch "master"]
    remote = origin
    merge = refs/heads/master
[remote "bithub"]
    url = git@bithub:bneuburg/volatility.git
    fetch = +refs/heads/*:refs/remotes/bithub/*
[remote "dwarf"]
    url = https://github.com/rcatolino/volatility.git
    fetch = +refs/heads/*:refs/remotes/dwarf/*
[branch "Linux4_8_kaslr_support"]
    remote = bithub
    merge = refs/heads/Linux4_8_kaslr_support

Oder:
$ git config -l
user.name=Bastian Neuburger
user.email=b.neuburger@gsi.de
user.signingkey=Bastian Neuburger
color.ui=auto
alias.ci=commit -S
alias.unstage=reset HEAD --
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
remote.origin.url=https://github.com/volatilityfoundation/volatility.git
branch.master.remote=origin
branch.master.merge=refs/heads/master
remote.bithub.url=git@bithub:bneuburg/volatility.git
remote.bithub.fetch=+refs/heads/*:refs/remotes/bithub/*
remote.dwarf.url=https://github.com/rcatolino/volatility.git
remote.dwarf.fetch=+refs/heads/*:refs/remotes/dwarf/*
branch.Linux4_8_kaslr_support.remote=bithub
branch.Linux4_8_kaslr_support.merge=refs/heads/Linux4_8_kaslr_support

  • git config -l liest nicht nur die lokale Repo-Config, sondern sonstige systemweite Einstellungen.
  • Um nur die fuers Repo zu bekommen, mach ich:
$ git config -l --local
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
remote.origin.url=https://github.com/volatilityfoundation/volatility.git
branch.master.remote=origin
branch.master.merge=refs/heads/master
remote.bithub.url=git@bithub:bneuburg/volatility.git
remote.bithub.fetch=+refs/heads/*:refs/remotes/bithub/*
remote.dwarf.url=https://github.com/rcatolino/volatility.git
remote.dwarf.fetch=+refs/heads/*:refs/remotes/dwarf/*
branch.Linux4_8_kaslr_support.remote=bithub
branch.Linux4_8_kaslr_support.merge=refs/heads/Linux4_8_kaslr_support

Ich sehe hier also alle meine Remotes und fuer einen Teil meiner Branches die Settings branch.$BRANCHNAME.{remote|merge}

In diesem Beispiel gilt nun folgendes:
  • bin ich auf dem Branch master und mache ein nackiges git push, will er das nach Remote origin pushen, also https://github.com/volatilityfoundation/volatility.git
  • bin ich auf dem Linux4_8_kaslr_support Branch, will er nach bithub, also git@bithub:bneuburg/volatility.git, pushen
  • auf anderen Branches will er nach origin pushen, da das ein built-in Default ist.

  • Neben nackigen git pushes gibt es noch 2 Varianten, naemlich
$ git push $REMOTE
  • Das forciert unabhaengig von der git config das angegebene Remote.

Statt einem Remote Alias zu nehmen, kannst du hier auch bisher undefinierte URLs als absolute URL eingeben. Beispiel:
$ mkdir /tmp/test
$ pushd /tmp/test
$ git init
Initialized empty Git repository in /tmp/test/.git/
$ popd # zurueck ins original Git Repo
$ git push file:///tmp/test
No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as 'master'.
fatal: The remote end hung up unexpectedly
error: failed to push some refs to 'file:///tmp/test'

Hmph! Der weigert sich einfach. Das liegt daran, dass beide Repos keine gemeinsamen "refs", also Historie oder Branches mit mehr als 0 commits haben.

In solchen Faellen (und genau die hast du wenn, du z.B. ueber github/gitlab/gitorious Web UI ein neues leeres Repo zusammen klickst) musst du noch den Branch angeben, den du pushen willst.

$ git push $REMOTE $BRANCH [$BRANCH2, $BRANCH3,...]

Im Beispiel:
$ git push file:///tmp/test Linux4_8_kaslr_support
Counting objects: 44, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (35/35), done.
Writing objects: 100% (35/35), 8.19 KiB, done.
Total 35 (delta 28), reused 0 (delta 0)
Unpacking objects: 100% (35/35), done.
To file:///tmp/test
 * [new branch]      Linux4_8_kaslr_support -> Linux4_8_kaslr_support

  • Hierbei ist zu beachten, dass durch das git init in /tmp/test ein leerer Master Branch angelegt wurde, auf den du by default nicht pushen kannst. Spielt aber auch keine Rolle, da das bei gitlab fuer dich geregelt wird, das Problem besteht nur im lokalen Beispiel.

  • Diese Form von git push musst du immer dann machen, wenn du einen lokalen Branch, den es bei nem Remote noch nicht gibt, dorthin pushen willst.

Quintessenz:

  • git push pushed den momentan ausgecheckten Branch nach branch.$BRANCH.remote, falls das nicht explizit gesetzt ist nach origin, und falls origin nicht definiert ist, meckert er.
  • git push $REMOTE wenn du nicht an den vordefinierten branch.$BRANCH.remote pushen willst.
  • git push $REMOTE $BRANCH wenn du nen anderen Branch an ein beliebiges Remote pushen willst oder der Branch im Remote noch nicht existiert.

Beachte: git push origin geht, wenn origin definiert ist, git push master geht nicht, auch wenn der master Branch existiert.

Unwilling remotes

Issue Although there is only one remote and only one branch in your repo, git push still does not work - you are forced to add more characters: git push origin master

Solution git branch --set-upstream-to=origin/master

Migration from gitorious.gsi.de to git.gsi.de (GitLab)

If your remote is called origin (the default):

git remote set-url origin git@git.gsi.de:group/project.git
git pull -r
gut push

Migration from SVN

git svn allows using git to work on a central SubVersion repository

Create a local git repo from the svn repository at https://svn.example.com/project:

$ git svn clone https://svn.example.com/project 

After import you can add a git remote and push it there

Updating from SVN

You can pull in upstream changes in SVN with
$ git svn fetch
$ git svn rebase

Pushing back to SVN

You can commit your local changes back to the central SVN with:
$ git svn dcommit

Advanced usage

Rewriting SVN authors

SVN may use different usernames than git (normally realname + email). These can be mapped during import with an authors file:

$ cat gitsvn.authors
userid: Real Name 
…
$ git svn clone --authors-file=gitsvn.authors https://svn.example.com/project

Branches and Tags

If you use the standard svn repo layout with subdirs for trunk/, branches/ and tags/ you can use the --stdlayout option with git svn clone.

If you use a different layout, you can specify the location of branches and/or tags with individual options --branches and --tags.

References

Topic revision: r24 - 2021-11-26, ThomasRoth
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding GSI Wiki? Send feedback | Legal notice | Privacy Policy (german)