Tutor Plugins

Getting started

Enable the plugin with:

tutor plugins enable cairn

Then, restart your platform and run the initialization scripts:

tutor local launch

Open http(s)://data.<YOUR_LMS_HOST> in your browser. When running locally, this will be http://data.local.edly.io. (http://data.local.edly.io:2247 in development). Users authenticate with their LMS user. By default, they have access to the data generated by the courses in which they have the "staff role". Once a user has successfully logged-in, their account can be modified with the cairn-createuser command.

For instance, to convert an existing user to administrator status, run:

tutor local do cairn-createuser --admin YOURUSERNAME YOURUSERNAME@YOUREMAIL.COM

To add the default dashboards to the new user, add the --bootstrap-dashboards option:

tutor local do cairn-createuser --bootstrap-dashboards YOURUSERNAME YOURUSERNAME@YOUREMAIL.COM

Some event data might be missing from your dashboards: just start using your LMS and refresh your dashboard. The new events should appear immediately.

By default, AUTH_ROLES_SYNC_AT_LOGIN is false which means admin can customize the permissions associated to a user. To disable this behaviour, modify the CAIRN_AUTH_ROLES_SYNC_AT_LOGIN setting:

tutor config save --set CAIRN_AUTH_ROLES_SYNC_AT_LOGIN=true
tutor local restart

⚠️ WARNING ⚠️ Previous versions (v15 and earlier) of Cairn required manual user management. If you have an existing installation of Cairn, this behaviour will change when you upgrade to v16. To revert to the previous behaviour, see "manual user management" below.

Available metrics

Cairn allows you to collect and view just any metric from your Open edX platform, as long as the data is available from the tracking logs or the MySQL database. Out of the box, the default Cairn dashboard comes with visualizations for the following metrics:

• Weekly course engagement:

  • Number of enrolled learners
  • Number of active learners
  • Number of learners who watched a video
  • Number of learners who tried a problem
  • Average time spent in course

• Course progress:

  • Number of students who completed each unit
  • Grade distribution histogram

• Course demographics:

  • Gender distribution
  • Level of education distribution

• Video engagement:

  • Number of unique viewers
  • Average watch time
  • Total watch time
  • Second-per-second statistics: Number of unique viewers, Total number of views

Manual user management

By default, authentication uses single sign-on (SSO) with the LMS such that users do not have to create separate accounts in Superset. In previous versions of Cairn (v15 and earlier), user accounts had to be created manually. To restore this behaviour, modify the CAIRN_ENABLE_SSO setting:

tutor config save --set CAIRN_ENABLE_SSO=false
tutor local restart

SSO will then disabled, and only manually created users will be able to login. To create a user, run the cairn-createuser command, as explained above:

tutor local do cairn-createuser --password=yourpassword YOURUSERNAME YOURUSERNAME@YOUREMAIL.COM

To restrict a given user to one or more courses or organizations, select the course IDs and/or organization IDS to which the user should have access:

tutor local do cairn-createuser --course-id='course-v1:edX+DemoX+Demo_Course' YOURUSERNAME YOURUSERNAME@YOUREMAIL.COM

Refreshing course block data

Course block IDs and names are loaded from the Open edX modulestore into the datalake. After making changes to your course, you might want to refresh the course structure stored in the datalake. To do so, run:

tutor local do init --limit=cairn

Or, if you want to avoid running the full plugin initialization:

tutor local run \
    -v $(tutor config printroot)/env/plugins/cairn/apps/openedx/scripts/:/openedx/scripts \
    -v $(tutor config printroot)/env/plugins/cairn/apps/clickhouse/auth.json:/openedx/clickhouse-auth.json \
    lms python /openedx/scripts/importcoursedata.py

Running on Kubernetes

When running on Kubernetes instead of locally, most commands above can be re-written with tutor k8s exec service "command" instead of tutor local run service command. For instance:

# Privileged user creation
tutor k8s exec cairn-superset "superset fab create-admin --username YOURUSERNAME --email user@example.com"
# Unprivileged user creation
tutor k8s exec cairn-clickhouse "cairn createuser --course-id='course-v1:edX+DemoX+Demo_Course' --org-id='edX' YOURUSERNAME"
tutor k8s exec cairn-superset "cairn createuser YOURUSERNAME YOURUSERNAME@YOUREMAIL.COM"

Collecting past events

When Cairn is launched for the first time, past events that were triggered prior to the plugin installation will not be loaded in the data lake. If you are interested in loading past events, you should load them manually by running:

tutor local start -d cairn-clickhouse
tutor local run \
  --volume="$(tutor config printroot)/data/lms/logs/:/var/log/openedx/:ro" \
  --volume="$(tutor config printroot)/env/plugins/cairn/apps/vector/file.toml:/etc/vector/file.toml:ro" \
  -e VECTOR_CONFIG=/etc/vector/file.toml cairn-vector

The latter command will parse tracking log events from the $(tutor config printroot)/data/lms/logs/tracking.log file that contains all the tracking logs since the creation of your platform. The command will take a while to run if you have a large platform that has been running for a long time. It can be interrupted at any time and started again, as the log collector keeps track of its position within the tracking log file.

General settings

• CAIRN_HOST (default: "data.{{ LMS_HOST }}"): hostname at which the Cairn frontend (i.e: Superset) will be accessible. By default, this is the "data" subdomain of the LMS. Thus, if your students access the LMS at https://learn.mydomain.com then Cairn will be accessible at https://data.learn.mydomain.com.
• CAIRN_DOCKER_HOST_SOCK_PATH (default: "/var/run/docker.sock"): path to the Docker host socket on the host. This is required to collect logs from Docker when running locally, but it is not used when running on Kubernetes.

Clickhouse settings

• CAIRN_RUN_CLICKHOUSE (default: true): set to false to run your own Clickhouse cluster separately from Cairn. In that case, you should also configure the Clickhouse credentials below.
• CAIRN_CLICKHOUSE_DOCKER_IMAGE (default: "{{ DOCKER_REGISTRY }}overhangio/cairn-clickhouse:{{ CAIRN_VERSION }}"): name of the Docker image that runs Clickhouse. Override this setting to build your own image of Clickhouse.
• CAIRN_CLICKHOUSE_HOST (default: "cairn-clickhouse"): hostname where Clickhouse will be accessible from Superset. By default, this is the internal docker-compose/Kubernetes service name.
• CAIRN_CLICKHOUSE_HTTP_PORT (default: 8123): port at which Clickhouse exposes its HTTP API, which is necessary to bulk import unit names.
• CAIRN_CLICKHOUSE_HTTP_SCHEME (default: "http"): HTTP scheme to access the Clickhouse HTTP API. If you self-host a Clickhouse cluster (RUN_CLICKHOUSE=false) then it is strongly recommended to set this to "https".
• CAIRN_CLICKHOUSE_PORT (default: 9000): native Clickhouse API port.
• CAIRN_CLICKHOUSE_DATABASE (default: "openedx"): name of the Clickhouse database which will store all analytics from your Open edX platform.
• CAIRN_CLICKHOUSE_USERNAME (default: "openedx"): username to access the CAIRN_CLICKHOUSE_DATABASE.
• CAIRN_CLICKHOUSE_PASSWORD (default: "{{ 20|random_string }}"): randomly-generated password for CAIRN_CLICKHOUSE_USERNAME.

Postgresql/Superset settings

• CAIRN_RUN_POSTGRESQL (default: true): set to false to run your own Postgresql cluster separately from Cairn. Postgresql is the database that stores all data related to Superset, which is the Cairn frontend.
• CAIRN_SUPERSET_LANGUAGE_CODE (default: "{{ LANGUAGE_CODE[:2] }}"): 2-letter code of the default language for the Superset frontend. View the list of all supported languages here. When different than "en", users will have the opportunity to switch from English to this language via a flag icon in the top-right corner.
• CAIRN_SUPERSET_DOCKER_IMAGE (default: "{{ DOCKER_REGISTRY }}overhangio/cairn-superset:{{ CAIRN_VERSION }}"): name of the Docker image that runs Postgresql.
• CAIRN_POSTGRESQL_DATABASE (default: "superset"): name of the Postgresql database.
• CAIRN_POSTGRESQL_USERNAME (default: "superset"): Postgresql username.
• CAIRN_POSTGRESQL_PASSWORD (default: "{{ 20|random_string }}"): Postgresql password.
• CAIRN_SUPERSET_SECRET_KEY (default: "{{ 20|random_string }}"): randomly-generated secret key for the Superset frontend.