Transparent Open edX development and server administration

This plugin for Tutor allows you to transparently manage multiple Tutor environments on remote servers. Don't know what that means? Trust me, it's really cool ;) Here are a few things you can do with Tutor Machine:

  • Edit code locally in your favorite IDE and see it run remotely
  • Develop Tutor plugins and Open edX themes on your laptop and immediately apply the changes in production.
  • Quickly build Docker images despite a poor bandwidth
  • Run a full-featured Open edX development environment on any lower end laptop
  • Manage multiple development and production environments without clogging your laptop
  • Run Tutor and Open edX without even having to install Docker or docker-compose on your computer

This work was inspired by the Sultan project made by the wonderful people at Appsembler.

Installation

This plugin is available to subscribers of a Tutor Wizard Edition license. To install the plugin, run:

tutor license install tutor-machine

Then, enable the plugin with:

tutor plugins enable machine

Usage

Launch a remote server

You should launch a remote server to which you can connect by ssh. This step is not handled by Tutor Machine.

For instance, to launch a DigitalOcean droplet, you can use the doctl CLI:

# Find the ID of an SSH key which you can use to connect to your droplet
doctl compute ssh-key list

# Launch a droplet
doctl compute droplet create \
    --size=s-4vcpu-8gb --region=ams3 --image=ubuntu-20-04-x64 \
    --ssh-keys=MYSSHKEYID --wait \
    myopenedxserver
Add your remote server to Tutor Machine

To add your connection credentials to Tutor Machine, run:

tutor machine add

Type the name you would like to give to this machine, then the SSH username (e.g: "root" or "ubuntu") and hostname (IP address or domain name).

To start using this server with Tutor, you should "activate" it, just like you would do with a Python virtual environment:

source $(tutor machine printroot)/servers/YOURSERVERNAME/activate

Note that this server will have to be activated again if you open a new shell. Personally I like to define a wotm ("work on tutor machine") bash alias in my ~/.bashrc:

wotm() {
    source $(tutor machine printroot)/servers/$1/activate
}

That way, I simply have to type wotm prod to start working on my "prod" server.

To stop using a server at any time, simply run deactivatetutormachine.

Bootstrap your server

Your server needs Docker and docker-compose to start running an Open edX platform. To install these, run:

tutor machine bootstrap

Note that installing Tutor on your remote server is not required. Tutor is run locally, and only the docker and docker-compose commands will be run remotely.

Synchronize local and remote environment

If your remote server already has a running Open edX platform, then you will want to fetch the Tutor environment stored there:

tutor machine sync --reverse

This will run rsync to fetch the tutor/env folder as well as the tutor/config.yml file stored on your remote machine. You can safely ignore this step if Tutor does not run on your remote machine. Environment and configuration changes will be synced automatically from your local computer to the remote machine every time you run tutor config save.

If you would rather edit files manually in your environment (or your bind-mounted volumes, see below), you can watch for changes:

tutor machine sync --watch

This will automatically run rsync every time a change occurs in one of the watched folders.

Start managing your remote platform

You can now run any tutor command to manage your remote platform just as if you were running it locally.

Build Docker images:

tutor images build

Start using any plugin:

tutor plugin enable MYPLUGIN
tutor config save

Launch a full Open edX platform:

tutor local quickstart
Use Tutor Machine for remote development

Among other things, you can run a development environment on your remote machine:

tutor dev runserver lms

The LMS will then be made available on port 8000 of your remote server. To access this port from your local computer, you should open up a tunnel to the remote server:

tutor machine tunnel 8000

Then, try to open http://local.overhang.io:8000 in a browser on your local computer: you should be able to see the LMS running on the remote machine.

Note that you can open multiple ports at once. For instance, to open both the ports for the LMS and the CMS:

tutor machine tunnel 8000 8001
Run arbitrary commands on the remote machine

To connect to the remote machine and execute just any command, run tutor machine exec. For instance, to view resource usage per container, run:

tutor machine exec docker starts
Edit code locally and run it remotely

With the recently introduced tutor dev bindmount command, it becomes possible to edit code locally and see the changes (almost) instantly on the remote host. For instance, to start working on edx-platform:

# Bind-mount the edx-platform folder on the remote
# The volumes/ directory, which contains the edx-platform
# copy, will be automatically synced to the local host.
tutor dev bindmount

# Watch for changes on the local host and sync them to the remote
tutor machine sync --watch bindmounts

# Run a dev server on the remote and create a tunnel to view the
# running platform at http://local.overhang.io:8000
tutor dev runserver --volume=/openedx/edx-platform lms
tutor machine tunnel 8000

# Edit some files and see your changes automatically
# go live at http://local.overhang.io:8000
vim $(tutor config printroot)/volumes/edx-platform/manage.py

To view the progress of the rsync commands, you can either pass the --verbose option to the sync commands, or define the TUTOR_MACHINE_SYNC_VERBOSE environment variable.

Server configuration

The machine configuration stored at $(tutor machine printroot)/servers/YOURSERVERNAME/config.yml can be manually modified with a simple text editor to add a few settings.

Extra SSH arguments

Sometimes it is necessary to pass extra arguments to the ssh command to connect to a remote machine. This is possible by adding an opts list to the ssh setting. For instance, to manually specify an SSH private key:

ssh:
  hostname: 192.165.27.12
  user: ubuntu
  opts:
    - "-i"
    - /path/to/identityfile/id_rsa
Tutor remote root

By default, we assume that the Tutor environment is stored on the remote machine at .local/share/tutor. If this is not the case, you can define a custom path in the server configuration file:

tutor_root: /path/to/custom/tutor/

License

This work is licensed under the terms of the GNU Affero General Public License (AGPL).