Tuesday, June 10, 2014

How to develop for Sage using Sage Math Cloud
 and Git

I will be using Sage Math Cloud to write all the code for my Google Summer of Code project. The advantage of choosing the cloud-based route should be clear: you don’t need to install or store anything locally, and it frees you up to use any device with a web browser you choose to write code. The downside, as Sage Math Cloud is still new, is that tutorials and how-tos on the setup process in order to do so are a bit lacking. Since I couldn’t find any single unified source on how to do it, I thought I might to create a step-by-step guide on getting set up in Sage Math Cloud to write code in for inclusion in the Sage codebase.

Much of what I've done follows the Git the Hard Way tutorial for Sage development. However, there are significant departures. I recommend reading through that tutorial first; if you do then what follows below will be a piece of cake.

First, you’ll need an account at cloud.sagemath.com. Good news: the whole service is completely free to use, and there’s no annoying back-and-forthing via emailed confirmation links. It’ll take you two minutes tops.

The cloud.sagemath.com homepage.

Your work on your Sage Math Cloud account is project-based; each account can have a large number of projects associated to it. Each project acts like it’s own Linux-based file system inside of which you can create and edit any number of files of any type. Create a project in which you'll house your code.

The project root view, before any files have been created.

Next, you'll want a local installation of Sage. We're going to initiate a local copy of the Sage master repo, as hosted on GitHub. Click New --> Terminal to open up a new terminal window - we'll be doing everything via terminal commands. Then type the following three commands:
~$ git clone git://github.com/sagemath/sage.git
~$ cd sage
~/sage$ make
The first will download a copy of the Sage codebase into the new folder 'sage' (it should appear in the project root directory after the download is complete); the second changes into the directory, and the third initiated the compiling of the Sage source.
Note that the building of Sage usually takes about 3 hours, so get this going and then kick back and take inspiration from this.

While your code is compiling you'll want to make sure you have a GitHub account. If you don't follow the steps there to create one. While not required, hosting any projects on GitHub is highly recommended: it will allow multiple people to work on a project simultaneously, as well as giving you access to sophisticated revision control functionality that will prevent you from accidentally deleting code or data.

GitHub repo creation page.

Create an empty repository - we'll fill it with things later. So long as the repo exists and you know its name you're good to go.

The next thing we'll want to do is register the Sage Trac repository, where all the active tickets are hosted for Sage development. Once you're done creating an empty repo on your GitHub account, go back to your Sage Math Cloud project and wait for Sage to finish building. When this is done, in the terminal window type the following:

~/sage$ git remote add trac git@trac.sagemath.org:sage.git -t master
~/sage$ git remote -v
origin  git://github.com/sagemath/sage.git (fetch)
origin  git://github.com/sagemath/sage.git (push)
trac    git@trac.sagemath.org:sage.git (fetch)
trac    git@trac.sagemath.org:sage.git (push)

The git remote add command registers the Sage trac repo as a remote repository (trac is the name it is registered as locally; you can of course call it anything you want). The git remote -v command simply lists what repos you have now registered. We won't be using Sage trac for now. If you want more practice with trac, see the Git the Hard Way tutorial mentioned previously.

Note the inclusion of the word master in the command above; this means we will only track the master branch of the remote repo. A branch is an instanced copy of the codebase which you can work on and edit without altering other branches. The master branch should be kept pristine, so we will want to create and switch to a new branch. This is accomplished with the git checkout command:
~/sage$ git checkout -b demo
Switched to a new branch 'demo'
~/sage$ git branch
The -b parameter creates the branch, while checkout switches to that branch. Typing git branch then shows you what branches currently exist locally, as well as the one you're currently on.

We're now in the position to register the demo branch with our fresh repo on GitHub:
~/sage$ git remote add demo https://github.com/haikona/demo.git
~/sage$ git remote -v
demo    https://github.com/haikona/demo.git (fetch)
demo    https://github.com/haikona/demo.git (push)
origin  git://github.com/sagemath/sage.git (fetch)
origin  git://github.com/sagemath/sage.git (push)
trac    git@trac.sagemath.org:sage.git (fetch)
trac    git@trac.sagemath.org:sage.git (push)
And finally, we can push to our repo to sync the local copy therewith. For this you will need your GitHub password - so not just any old Harry can push to your repo.

~/sage$ git push demo demo
Username for 'https://github.com': haikona
Password for 'https://haikona@github.com':
Counting objects: 4, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 317 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
To https://github.com/haikona/demo.git
   2b1d88b..21cddb8  demo -> demo

You're now synced and ready to start writing code! From hereon you should be able to follow one of the many tutorials on working with git; the hard part - setting up - is all done.


  1. I'm still at the point where I don't totally get git, so here's my question. When I run that last command, 'git push demo demo', instead of counting 4 objects, my git counts 218470 of them. It then goes ahead and pushes what appears to be the entire sage bundle onto my remote repository. Have I done something wrong? I hope so, because it seems to me like it would be more elegant if my remote was simply keeping track of files that were changed since the base sage install. Is that possible?

  2. I think that's correct behaviour (I'm in the same bucket as you in that I'm still learning the ropes with git too). Your repo should track the entire Sage install, so the initial push might be a bit meaty. However, once you set up a branch for development (and make that branch your default), any changes from thereon are localized to that branch, and only the files that you create or edit will need to be pushed and pulled. At least, this is what I did, and it seems to be working well for me.

  3. You can simplify this in general by clicking on the "Fork" button on Github at sagemath/sage. Then, your fork of Sage sits in haikona/sage and has the same naming and so on. Then (what I would be doing) is this remote mapping: "origin" -> "haikona/sage" and "upstream" -> "sagemath/sage".

    In usual circumstances, you would be able to use the pull request feature of Github, but it's Sage -> hence you actually need to have an account on trac and push your modifications in a new branch over there to a ticket. Behind the scenes it's the same, the interface is different and pull requests are then pointers to your new branch attached to a full ticket.

  4. Just to clarify, a git repo is always the entire history of a project. This is where the D=decentralized of DVCS comes from.

    Up/Downloading with git always only transfers changes, so if you start with an empty repo then the first "git push" must upload the whole repo. If you start with a fork then only differences relative to when you made the fork must be uploaded.

  5. Article is giving really productive information to everyone. Well done.You can also read more aboutSage Hosting on Techarex Networks Website.