Get the Code

Set Up Forks

If you’re contributing or working a lot on a feature, you’ll probably want your own forks and a slightly different git remote setup.

You can create forks of each Brooklyn repo in the GitHub UI or, if you have the command-line tool hub (described here, or sudo npm install -g hub), by running this command:

hub fork; git submodule foreach 'hub fork'

The Get the Code: Basics page described how to retrieve the upstream repos, but it gave those remotes the name origin. When using forks, upstream is a more accurate name. You can rename the origin remotes with:

git remote rename origin upstream; git submodule foreach 'git remote rename origin upstream'

You’ll now likely want to add the remote origin for your fork:

if [ -z "$GITHUB_ID" ] ; then echo -n "Enter your GitHub ID id: " ; read GITHUB_ID ; fi
git remote add origin git@github.com:${GITHUB_ID}/brooklyn
git submodule foreach 'git remote add origin git@github.com:${GITHUB_ID}/${name}'

And if you created the fork in the GitHub UI, you may want to create a remote named by your GitHub ID as well (if you used hub it will have done it for you):

if [ -z "$GITHUB_ID" ] ; then echo -n "Enter your GitHub ID id: " ; read GITHUB_ID ; fi
git remote add ${GITHUB_ID} git@github.com:${GITHUB_ID}/brooklyn
git submodule foreach 'git remote add ${GITHUB_ID} git@github.com:${GITHUB_ID}/${name}'

You probably also want the default push target to be your repo in the origin remote:

git config remote.pushDefault origin; git submodule foreach 'git config remote.pushDefault origin'

Optionally, if you’re interested in reviewing pull requests, you may wish to have git automatically check out PR branches:

git config --local --add remote.upstream.fetch '+refs/pull/*/head:refs/remotes/upstream/pr/*'
git submodule foreach "git config --local --add remote.upstream.fetch '+refs/pull/*/head:refs/remotes/upstream/pr/*'"
git pull ; git submodule foreach 'git pull'

And also optionally, to set up the official Apache repo as a remote — useful if GitHub is slow to update (and required if you’re a committer):

git remote add apache-git https://gitbox.apache.org/repos/asf/brooklyn
git submodule foreach 'git remote add apache-git https://gitbox.apache.org/repos/asf/${name}'

That’s it. Test that it’s all working by browsing the submodules and issuing git remote -v and git pull commands. Also see the aliases below.

To work on code in a branch, in any of the submodules, you can simply do the following:

% git branch my-new-feature-branch upstream/master
% git checkout my-new-feature-branch
(make some commits)
% git push
To https://github.com/your_account/brooklyn.git
 * [new branch]      my-new-feature-branch -> my-new-feature-branch

Note how the branch is tracking upstream/master for the purpose of git pull, but a git push goes to the fork. When you’re finished, don’t forget to go to the UI of your repo to open a pull request.

Multi-Project Changes

Cross-project changes will require multiple PRs: try to minimise these, especially where one depends on another, and especially especially where two depend on each other – that is normally a sign of broken backwards compatibility! Open the PRs in dependency order and assist reviewers by including the URLs of any upstream dependency PRs in the dependent PR to help reviewers (dependency PRs will then include a “mention” comment of the dependent PR).

For information on reviewing and committing PRs, see the committer’s guide.

How We Use Branches

History, Tags, and Workflow

There are branches for each released version and tags for various other milestones.

As described in more detail here, we primarily use submodule remote branch tracking rather than submodule SHA1 ID’s.

The history prior to 0.9.0 is imported from the legacy incubator-brooklyn repo for reference and history only. Visit that repo to build those versions; they are not intended to build here. (Although this works: mkdir merged ; for x in brooklyn-* ; do pushd $x ; git checkout 0.8.0-incubating ; cp -r * ../merged ; popd ; cd merged ; mvn clean install.)

Tracking Branches

Our submodules track branches, rather than specific commits, although due to the way git works there are still references to specific commits.

We track master for the master branch and the version branch for other official branches, starting with 0.9.0. We update the uber-project recorded SHA reference to subprojects on releases but not regularly – that just creates noise and is unnecessary with the --remote option on submodule update. In fact, git submodule update --remote --merge pretty much works well; the git sup alias (below) makes it even easier.

On the other hand, git status is not very nice in the uber-project: it will show a “new commits” message for submodules, unless you’re exactly at the uber-project’s recorded reference. Ignore these. It will tell you if you have uncommitted changes, but it’s not very useful for telling whether you’re up to date or if you have newer changes committed in the subproject or in your push branch. If you go in to each sub-project, git status works better, but it can be confusing to track which branch each subproject is one. A summary script is provided below which solves these issues, showing useful status across all subprojects.

About Submodules

Submodules can be confusing; if you get stuck the info and references in this section may be useful. You can also work without submodules.

Pitfalls of Submodules

Some of the things to be careful of are:

• Don’t copy submodule directories. This doesn’t act as you’d expect; its .git record simply points at the parent project’s .git folder, which in turn points back at it. So if you copy it and make changes in the copy, it’s rather surprising where those changes actually get made. Worse, git doesn’t report errors; you’ll only notice it when you see files change bizarrely.

• Be careful committing in the uber-project. You can update commit IDs, but if these accidentally point to an ID that isn’t committed, everyone else sees errors. It’s useful to do this on release (and update the target branch then also) and maybe occasionally at other milestones but so much at other times as these ID’s very quickly become stale on master.

Git Submodule References

• [1] Git SCM Book
• [2] Medium blog: Mastering Git Submodules
• [3] git submodule --help section on update
• [4] StackOverflow: Git Submodules Branch Tag

Not Using Submodules

If you don’t want to use submodules, you can clone everything as top-level projects with the following:

mkdir apache-brooklyn
cd apache-brooklyn
git clone http://github.com/apache/brooklyn/
git clone http://github.com/apache/brooklyn-ui/
git clone http://github.com/apache/brooklyn-server/
git clone http://github.com/apache/brooklyn-client/
git clone http://github.com/apache/brooklyn-docs/
git clone http://github.com/apache/brooklyn-library/
git clone http://github.com/apache/brooklyn-dist/

With one symbolic link in the root apache-brooklyn/ dir, you can then use a normal mvn workflow:

ln -s brooklyn/pom.xml .
mvn clean install

With minor changes you can follow the instructions for creating forks and getting all updates elsewhere on this page.

Useful Aliases and Commands

This sets up variants of pull, diff, and push – called sup, sdiff, and spush – which act across submodules:

# update all modules
git config --global alias.sup '!git pull && git submodule update --remote --merge --recursive'
# show diffs across all modules
git config --global alias.sdiff '!git diff && git submodule foreach "git diff"'
# return to master in all modules
git config --global alias.smaster '!git checkout master && echo && git submodule foreach "git checkout master && echo"'
# push in all modules
git config --global alias.spush '!git push && git submodule foreach "git push"'
# show issues in all projects (only works if upstream configured properly for current branch)
git config --global alias.si '!hub issue && git submodule foreach "hub issue"'

Getting a Summary of Submodules

The git-summary script in the brooklyn-dist/scripts makes working with submodules much more enjoyable. Follow the README in that directory to add those scripts to your path, and then set up the following git aliases:

curl https://gist.githubusercontent.com/ahgittin/6399a29df1229a37b092/raw/208cf4b3ec2ede77297d2f6011821ae62cf9ac0c/git-summary.sh \
  | sudo tee /usr/local/bin/git-summary
sudo chmod 755 /usr/local/bin/git-summary  
git config --global alias.ss '!git-summary -r'
git config --global alias.so '!git-summary -r -o'

Then git ss will give output such as:

brooklyn: master <- upstream/master (up to date)

brooklyn-client: master <- upstream/master (up to date)

brooklyn-dist: master <- upstream/master (up to date)

brooklyn-docs: master <- upstream/master (uncommitted changes only)
  M guide/dev/code/submodules.md

brooklyn-library: master <- upstream/master (up to date)

brooklyn-server: master <- upstream/master (up to date)

brooklyn-ui: test <- origin/test (upstream 2 ahead of master)
  > 62c553e Alex Heneveld, 18 minutes ago: WIP 2
  > 22cd0ad Alex Heneveld, 62 minutes ago: WIP 1
 ?? wip-local-untracked-file

The command git so does the same thing without updating remotes. Use it if you want it to run fast, or if you’re offline. For more information un git-summary --help.

Other Handy Commands

# run a git command (eg pull) in each submodule
git submodule foreach 'git pull'

# iterate across submodules in bash, e.g. doing git status
for x in brooklyn-* ; do pushd $x ; git status ; popd ; done

Legacy Incubator Pull Requests

If you need to apply code changes made pre-graduation, against the incubator repository, splitting it up into submodules, it’s fairly straightforward:

1. In the incubator codebase, start at its final state: cd .../incubator-brooklyn && git checkout master && git pull
2. Make a branch for your merged changes: git checkout -b my-branch-merged-master
3. Merge or rebase the required commits (resolving conflicts; but don’t worry about commit messages): git merge my-branch
4. Create a patch file: git diff > /tmp/diff-for-my-branch
5. Go to the new brooklyn uber-project directory. Ensure you are at master and all subprojects updated: cd .../brooklyn/ && git sup
6. Apply the patch: patch -p1 < /tmp/diff-for-my-branch
7. Inspect the changes: git ss
8. Test it, commit each changed project on a branch and create pull requests. Where applicable, record the original author(s) and message(s) in the commit.