From a4a85ce1b8e53937cc01199d63e659e0e4652d2b Mon Sep 17 00:00:00 2001 From: Tim Schubert Date: Thu, 20 Sep 2018 13:29:40 +0200 Subject: [PATCH 1/2] Add more detailed instructions to README.md --- README.md | 136 +++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 99 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index 7605968..9c4a346 100644 --- a/README.md +++ b/README.md @@ -1,53 +1,115 @@ # The abgabesystem -## Setup +## About -Operations 1 and 2 require super user privileges to the API. The rest don't. +*Behold, the (almighty) abgabesystem!** -1. Import the students participating in the course into Gitlab. This is required to assign projects to each student. If you have exported a list of groups and functions from Stud.IP you can use that. -``` -# abgabesystem users -s -b -p main -``` +The aim of this project is to automate the handling of students' homework +solutions using Gitlab. So far It can -2. Create a group for your course using -``` -# abgabesystem courses -c - ``` - -3. Create a fork of this project inside the namespace of the group that has been created and configure your API token (`PRIVATE_API_TOKEN`) and deploy key (`SSH_PRIVATE_KEY`) (see .gitlab-ci.yml) for the forked project. +- import student accounts from LDAP, +- import a list of users from Stud.IP, +- create groups for courses in Gitlab, +- set up repositories for the students +- run automated style-checks and +- test for plagiarisms. -4. Set up the project for the example solutions and the student projects. If you have pre-existing example solutions place them in `/solutions/solutions`. -``` -# abgabesystem projects -c -d -s -``` +## Setup Gitlab and CI runners -5. Add all administrative users (e.g. users supervising the course or checking homework solutions) to the group of the course. +There are multiple components involved in the abgabesystem. The CI script uses a +[https://github.com/timschubert/docker-abgabesystem](Docker Container) that +contains the Python module and the [https://jplag.ipd.kit.edu/](JPlag) +plagiarism checker. Another container with +[https://github.com/timschubert/docker-checkstyle](Checkstyle) is optionally +required for style checking of each student repository. -6. At the deadline of each exercise trigger the plagiarism checker using -``` -# git tag -# git push --tags -``` -It can be useful to do this from a cronjob. +If you do not already have a working Gitlab instance see +[https://docs.gitlab.com/omnibus/README.html#installation](here) how to install +and configure it. Additionally you will need the +[https://docs.gitlab.com/runner/](Gitlab CI runner). For performance reasons, +you might want to have the CI runner on another host than Gitlab or otherwise +limit the resources available to the runner (depending on the number of students +and CI jobs). -## Recommended settings for gitlab.rb +See [https://docs.gitlab.com/ce/administration/auth/ldap.html#doc-nav](here) on +how to configure LDAP authentication. + +## Install the python module + +Install the python module using ``` - gitlab_rails['gitlab_default_can_create_group'] = false - - # see gitlab documentation and add your ldap config - gitlab_rails['ldap_enabled'] = true +$ virtualenv abgabesystem +$ source abgabesystem/bin/activate +$ pip install . ``` -Also, you should +## Set up the course -- set the default project limit for each user to 0 -- set default settings for projects to partially protected so that developers (e.g. students) can not force push tags and commits to protected branches (master) which is important for plagiarism controls. +To proceed, you need to have an +[https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html](API token) +with administrative privileges. After having configured Gitlab and the runner, +continue with setting up your course. Gitlab can only add existing users to +projects, so we create pseudo-users that later will be fetched from LDAP, the +first time each user logs in. -## Workflow +First create your course either using the Gitlab UI or -To trigger the deadline of an exercise (e.g. Sunday at 15:00), push a tag (e.g. -ex1) to the cloned abgabesystem project. -The abgabesystem's CI job creates a tag of this name inside each student's project and then creates a checkout of each project's repository and runs [JPlag](https://github.com/jplag/jplag) to check for plagiates. -The results can be found inside the job artifacts. -The results are saved for each tag and can be downloaded as an archive. +``` +$ abgabesystem courses -c +``` + +Next, since there is currently no API available to export a list of participants +from [https://www.studip.de/](Stud.IP), we use the CSV file (encoded as latin-1 +🤢) that lists all students currently enrolled in the course. This list may of +course change from time to time, so make sure to re-run the script regularly. + +``` +$ abgabesystem users -s -b -p main +``` + +Now create a fork of this repository inside the namespace of the course. + +This repository contains CI jobs that need their own +[https://github.com/timschubert/docker-abgabesystem](Docker Container). Build +the container, push it to the container registry and create a new runner that +uses the container. You can also +[https://docs.gitlab.com/ce/ci/docker/using_docker_build.html](automate this) +using the CI scripts included in the Docker container projects and let your +Gitlab CI build and deploy the updated containers for you. + +Proceed by creating an API token that has access to the group of the course. Add +this token as `PRIVATE_API_TOKEN` to the +[https://docs.gitlab.com/ce/ci/variables/](secret variables) of the forked +abgabesystem project. Then generate an SSH deploy key and add the private part +as `SSH_PRIVATE_KEY` to the secret variables. The key will be used by the CI +script to fetch from the student projects. + +Set up the the student projects using + +``` +$ abgabesystem projects -c -d -s + +``` + +Here deploy_key is the *PUBLIC* part of the deploy key. This creates one +project per student, that each is forked from a project where your example +solutions will be stored (`/solutions/solutions`). Check that +`/solutions/` a contains project for each student. + +At last, you can add everyone with permission to view all student solutions to +the group of the course. + +## Checking student solutions + +When you have reachd the deadline for an exercise, push a new tag to +`/abgabesystem` to trigger the plagiarism checker and automatically +create a tag in each student project. + +``` +$ git tag +$ git push --tags +``` + +Check the build artifacts of the CI job for the results of the plagiarism +checker. From e48e7c693af8dde58cfa2104d84cbe173aa4af6f Mon Sep 17 00:00:00 2001 From: Tim Schubert Date: Thu, 20 Sep 2018 13:48:52 +0200 Subject: [PATCH 2/2] Fixup: Links and newlines --- README.md | 78 +++++++++++++++++++------------------------------------ 1 file changed, 27 insertions(+), 51 deletions(-) diff --git a/README.md b/README.md index 9c4a346..197a6b1 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,8 @@ *Behold, the (almighty) abgabesystem!** -The aim of this project is to automate the handling of students' homework -solutions using Gitlab. So far It can +The aim of this project is to automate the handling of students' homework solutions using Gitlab. +So far It can - import student accounts from LDAP, - import a list of users from Stud.IP, @@ -16,23 +16,15 @@ solutions using Gitlab. So far It can ## Setup Gitlab and CI runners -There are multiple components involved in the abgabesystem. The CI script uses a -[https://github.com/timschubert/docker-abgabesystem](Docker Container) that -contains the Python module and the [https://jplag.ipd.kit.edu/](JPlag) -plagiarism checker. Another container with -[https://github.com/timschubert/docker-checkstyle](Checkstyle) is optionally -required for style checking of each student repository. +There are multiple components involved in the abgabesystem. +The CI script uses a [Docker Container](https://github.com/timschubert/docker-abgabesystem) that contains the Python module and the [https://jplag.ipd.kit.edu/](JPlag) plagiarism checker. +Another container with [Checkstyle](https://github.com/timschubert/docker-checkstyle) is optionally required for style checking of each student repository. -If you do not already have a working Gitlab instance see -[https://docs.gitlab.com/omnibus/README.html#installation](here) how to install -and configure it. Additionally you will need the -[https://docs.gitlab.com/runner/](Gitlab CI runner). For performance reasons, -you might want to have the CI runner on another host than Gitlab or otherwise -limit the resources available to the runner (depending on the number of students -and CI jobs). +If you do not already have a working Gitlab instance see [here](https://docs.gitlab.com/omnibus/README.html#installation) how to install and configure it. +Additionally you will need the [Gitlab CI runner](https://docs.gitlab.com/runner/). +For performance reasons, you might want to have the CI runner on another host than Gitlab or otherwise limit the resources available to the runner (depending on the number of students and CI jobs). -See [https://docs.gitlab.com/ce/administration/auth/ldap.html#doc-nav](here) on -how to configure LDAP authentication. +See [here](https://docs.gitlab.com/ce/administration/auth/ldap.html#doc-nav) on how to configure LDAP authentication. ## Install the python module @@ -46,12 +38,9 @@ $ pip install . ## Set up the course -To proceed, you need to have an -[https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html](API token) -with administrative privileges. After having configured Gitlab and the runner, -continue with setting up your course. Gitlab can only add existing users to -projects, so we create pseudo-users that later will be fetched from LDAP, the -first time each user logs in. +To proceed, you need to have an [API token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with administrative privileges. +After having configured Gitlab and the runner, continue with setting up your course. +Gitlab can only add existing users to projects, so we create pseudo-users that later will be fetched from LDAP, the first time each user logs in. First create your course either using the Gitlab UI or @@ -59,10 +48,8 @@ First create your course either using the Gitlab UI or $ abgabesystem courses -c ``` -Next, since there is currently no API available to export a list of participants -from [https://www.studip.de/](Stud.IP), we use the CSV file (encoded as latin-1 -🤢) that lists all students currently enrolled in the course. This list may of -course change from time to time, so make sure to re-run the script regularly. +Next, since there is currently no API available to export a list of participants from [https://www.studip.de/](Stud.IP), we use the CSV file (encoded as latin-1 🤢) that lists all students currently enrolled in the course. +This list may of course change from time to time, so make sure to re-run the script regularly. ``` $ abgabesystem users -s -b -p main @@ -70,20 +57,14 @@ $ abgabesystem users -s -b -p main Now create a fork of this repository inside the namespace of the course. -This repository contains CI jobs that need their own -[https://github.com/timschubert/docker-abgabesystem](Docker Container). Build -the container, push it to the container registry and create a new runner that -uses the container. You can also -[https://docs.gitlab.com/ce/ci/docker/using_docker_build.html](automate this) -using the CI scripts included in the Docker container projects and let your -Gitlab CI build and deploy the updated containers for you. +This repository contains CI jobs that need their own [Docker Container](https://github.com/timschubert/docker-abgabesystem). +Build the container, push it to the container registry and create a new runner that uses the container. +You can also [automate this](https://docs.gitlab.com/ce/ci/docker/using_docker_build.html) using the CI scripts included in the Docker container projects and let your Gitlab CI build and deploy the updated containers for you. -Proceed by creating an API token that has access to the group of the course. Add -this token as `PRIVATE_API_TOKEN` to the -[https://docs.gitlab.com/ce/ci/variables/](secret variables) of the forked -abgabesystem project. Then generate an SSH deploy key and add the private part -as `SSH_PRIVATE_KEY` to the secret variables. The key will be used by the CI -script to fetch from the student projects. +Proceed by creating an API token that has access to the group of the course. +Add this token as `PRIVATE_API_TOKEN` to the [secret variables](https://docs.gitlab.com/ce/ci/variables/) of the forked abgabesystem project. +Then generate an SSH deploy key and add the private part as `SSH_PRIVATE_KEY` to the secret variables. +The key will be used by the CI script to fetch from the student projects. Set up the the student projects using @@ -92,24 +73,19 @@ $ abgabesystem projects -c -d -s ``` -Here deploy_key is the *PUBLIC* part of the deploy key. This creates one -project per student, that each is forked from a project where your example -solutions will be stored (`/solutions/solutions`). Check that -`/solutions/` a contains project for each student. +Here deploy_key is the *PUBLIC* part of the deploy key. +This creates one project per student, that each is forked from a project where your example solutions will be stored (`/solutions/solutions`). +Check that `/solutions/` a contains project for each student. -At last, you can add everyone with permission to view all student solutions to -the group of the course. +At last, you can add everyone with permission to view all student solutions to the group of the course. ## Checking student solutions -When you have reachd the deadline for an exercise, push a new tag to -`/abgabesystem` to trigger the plagiarism checker and automatically -create a tag in each student project. +When you have reachd the deadline for an exercise, push a new tag to `/abgabesystem` to trigger the plagiarism checker and automatically create a tag in each student project. ``` $ git tag $ git push --tags ``` -Check the build artifacts of the CI job for the results of the plagiarism -checker. +Check the build artifacts of the CI job for the results of the plagiarism checker.