Add note about default permission

Fix/subgroups

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,35 @@
+---
+name: Bug report
+about: Create a report to help us improve
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -1,3 +1,12 @@
+env/
 List_of_groups_Lecture_*.csv
 __pycache__/
 _build/
+.eggs/
+.pytest_cache/
+.tox/
+dist/
+input/
+results/
+solutions/
+src/abgabesystem.egg-info/

.gitlab-ci.yml

@@ -8,11 +8,13 @@ variables:
 stages:
   - test
   - doc
+  - projects
   - deadline
   - plagiates
 
 before_script:
   ## get ssh private key from secret variable
+  - echo "$SSH_PUBLIC_KEY" | tr -d '\r' > deploy_key.pub
   - eval $(ssh-agent -s)
   - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
   - mkdir -p ~/.ssh
@@ -21,8 +23,8 @@ before_script:
   - ssh-keyscan $CI_REPO_HOST | tee ~/.ssh/known_hosts
   - chmod 644 ~/.ssh/known_hosts
   ## get API token from secret variable
+  - ./tools/build-config.sh 
   - cp python-gitlab.cfg $HOME/.python-gitlab.cfg
-  - echo "private_token = ${PRIVATE_API_TOKEN}" >> $HOME/.python-gitlab.cfg
   - python3 setup.py install
 
 deadlines:
@@ -61,7 +63,7 @@ doc:
     - abgabesystem
 
   script:
-    - cd doc && make html
+    - cd docs && make html
 
   artifacts:
     paths:
@@ -69,3 +71,15 @@ doc:
 
   only:
     - master
+
+create_projects:
+  ## create projects for all enrolled students
+  stage: projects
+  tags:
+    - abgabesystem
+
+  script:
+    - abgabesystem projects -c $CI_PROJECT_NAMESPACE -d deploy_key.pub
+
+  only:
+    - branches

README.md

@@ -1,53 +1,86 @@
 # The abgabesystem
 
-## Setup
+[GitHub](https://github.com/timschubert/abgabesystem)
 
-Operations 1 and 2 require super user privileges to the API. The rest don't.
+## About
 
-1. Import the students participating in the course into Gitlab. This is required to assign projects to each student. If you have exported a list of groups and functions from Stud.IP you can use that.
-```
-# abgabesystem users -s <students.csv> -b <LDAP base domain> -p main
-```
+*Behold, the (almighty) abgabesystem!*
 
-2. Create a group for your course using
-```
-# abgabesystem courses -c <some_course>
-   ```
+The aim of this project is to automate the handling of students' homework solutions using Gitlab.
+So far It can
 
-3. Create a fork of this project inside the namespace of the group that has been created and configure your API token (`PRIVATE_API_TOKEN`) and deploy key (`SSH_PRIVATE_KEY`) (see .gitlab-ci.yml) for the forked project.
+- import student accounts from LDAP
+- import a list of users from Stud.IP
+- create groups for courses in Gitlab
+- set up repositories for the students
+- run automated style-checks
+- test for plagiarisms
 
-4. Set up the project for the example solutions and the student projects. If you have pre-existing example solutions place them in `<some_course>/solutions/solutions`.
-```
-# abgabesystem projects -c <some_course> -d <deploy key> -s <students.csv>
-```
+## Setup Gitlab and CI runners
 
-5. Add all administrative users (e.g. users supervising the course or checking homework solutions) to the group of the course.
+There are multiple components involved in the abgabesystem.
+The CI script uses a [Docker Container](https://github.com/timschubert/docker-abgabesystem) that contains the Python module and the [JPlag](https://jplag.ipd.kit.edu/) plagiarism checker.
+Another container with [Checkstyle](https://github.com/timschubert/docker-checkstyle) is optionally required for style checking of each student repository.
 
-6. At the deadline of each exercise trigger the plagiarism checker using
-```
-# git tag <exercise_name>
-# git push --tags
-```
-It can be useful to do this from a cronjob.
+If you do not already have a working Gitlab instance see [here](https://docs.gitlab.com/omnibus/README.html#installation) how to install and configure it.
+Additionally you will need the [Gitlab CI runner](https://docs.gitlab.com/runner/).
+For performance reasons, you might want to have the CI runner on another host than Gitlab or otherwise limit the resources available to the runner (depending on the number of students and CI jobs).
 
-## Recommended settings for gitlab.rb
+See [here](https://docs.gitlab.com/ce/administration/auth/ldap.html#doc-nav) on how to configure LDAP authentication.
+
+## Install the python module
+
+Install the python module using
 
 ```
- gitlab_rails['gitlab_default_can_create_group'] = false
-
- # see gitlab documentation and add your ldap config
- gitlab_rails['ldap_enabled'] = true
+$ virtualenv abgabesystem
+$ source abgabesystem/bin/activate
+$ pip install .
 ```
 
-Also, you should 
+## Set up the course
 
-- set the default project limit for each user to 0
-- set default settings for projects to partially protected so that developers (e.g. students) can not force push tags and commits to protected branches (master) which is important for plagiarism controls.
+To proceed, you need to have an [API token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with administrative privileges.
+After having configured Gitlab and the runner, continue with setting up your course.
+Gitlab can only add existing users to projects, so we create pseudo-users that later will be fetched from LDAP, the first time each user logs in.
 
-## Workflow
+First create your course either using the Gitlab UI or
 
-To trigger the deadline of an exercise (e.g. Sunday at 15:00), push a tag (e.g.
-ex1) to the cloned abgabesystem project.
-The abgabesystem's CI job creates a tag of this name inside each student's project and then creates a checkout of each project's repository and runs [JPlag](https://github.com/jplag/jplag) to check for plagiates.
-The results can be found inside the job artifacts.
-The results are saved for each tag and can be downloaded as an archive.
+```
+$ abgabesystem courses -c <some_course>
+```
+
+Next, since there is currently no API available to export a list of participants from [https://www.studip.de/](Stud.IP), we use the CSV file (encoded as latin-1 🤢) that lists all students currently enrolled in the course.
+This list may of course change from time to time, so make sure to re-run the script regularly.
+
+```
+$ abgabesystem users -c <course> -s <students.csv> -b <LDAP base domain> -p main
+```
+
+Now create a fork of this repository inside the namespace of the course.
+
+This repository contains CI jobs that need their own [Docker Container](https://github.com/timschubert/docker-abgabesystem).
+Build the container, push it to the container registry and create a new runner that uses the container.
+You can also [automate this](https://docs.gitlab.com/ce/ci/docker/using_docker_build.html) using the CI scripts included in the Docker container projects and let your Gitlab CI build and deploy the updated containers for you.
+
+Proceed by creating an API token that has access to the group of the course.
+Add this token as `PRIVATE_API_TOKEN` to the [secret variables](https://docs.gitlab.com/ce/ci/variables/) of the forked abgabesystem project.
+Then generate an SSH deploy key and add the private part as `SSH_PRIVATE_KEY` and the public key as `SSH_PUBLIC_KEY` to the secret variables.
+The key will be used by the CI script to fetch from the student projects.
+
+At last, you can add everyone with permission to view all student solutions to the group of the course.
+
+## Permissions
+
+Configure Gitlab to allow developers to push on the master branch, but not force push to protected branches. An easy way to achieve this is to set Gitlab to "Partially Protected". A sane default is also to not allow students to create new projects.
+
+## Checking student solutions
+
+When you have reachd the deadline for an exercise, push a new tag to `<course>/abgabesystem` to trigger the plagiarism checker and automatically create a tag in each student project.
+
+```
+$ git tag <exercise_name>
+$ git push --tags
+```
+
+Check the build artifacts of the CI job for the results of the plagiarism checker.

config.yml

@@ -1,9 +0,0 @@
-ldap:
-  base: 'ou=people,dc=tu-bs,dc=de'
-  provider: main
-course:
-  !!python/object:abgabesystem.Course
-  name: test_course
-  students: Students.csv
-  deploy_key:
-    'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKl2zu3ClMIOI6EhEi0qGjwCgEaWYfRl2149T45pcggnYc3CVln0FJhjXvWbfMU984TjJMw4X8dfeZpf9p7xtieAab6yz+vB6QTW1ur9Uge0Wv/D084Sdzb3FovC+Qr90d6BAd+A6+v/vEprTLnuX8McQuB4p8l6iimFrhmv4IdrD1W/y0AUEzdz/eXpsHavlqGrpb4oQ0aAnZq0qQ9cYAltcXKQzgLi7zoKJGNWR+gz4hfRfqme87+k0ABO3hWwcIuwm/XdHm9Z+hjZrPfqmZGJF71FasE9jymP0Si4sgQLjaX+qQh3ojubBN7RwhUo3zjFFFUL5/tLEIr42SGpXF abgabesystem'

doc/notes.md

@@ -1,63 +0,0 @@
-# Programmieren [1,2] Gitlab
-
-- https://docs.gitlab.com/omnibus/README.html
-
-## Authentication
-
-- use GITZ LDAP for login
-- not allow "create new repo"
-
-
-## Structure
-
-- main repo
-  + publish example solutions
-  + CI config for checkstyle
-  + Protected Runner for JPlag
-  + restrict access to branches with example solutions
-   
-- student repos
-  + forked from main repo
-  + one repo per student
-  + student has *Developer* Access
-  + *tutors* group has *Master* access
-  + students can request access (Abgabepartner)
-  + *tutors* can grant access
-
-## Checkstyle
-
-- GitLab CI
-- [Docker](https://docs.gitlab.com/omnibus/docker/README.html)container
-- [Shared Runner](https://docs.gitlab.com/ce/ci/runners/README.html)
-- restrict Container to [checkstyle](http://checkstyle.sourceforge.net/)
-- disable internet access for container
-  
-## JPlag
-
-- Deadline [at,cron]job or schedule via gitlab
-- triggers [Protected Runner](https://docs.gitlab.com/ee/ci/runners/README.html#protected-runners)
-- creates automatic protected TAG in each repo
-- checks out TAG from all repos into /tmp and runs [JPlag](https://jplag.ipd.kit.edu/)
-- replace with MOSS? https://github.com/soachishti/moss.py
-- deploy key in each repo
-
-## (optional) sync script
-
-- (one-way) sync students and groups from [Stud.IP REST API](http://docs.studip.de/develop/Entwickler/RESTAPI) to [Gitlab REST API](https://docs.gitlab.com/ce/api/)
-
-# Replicate (TODO: ansible playbook)
-
-- install gitlab
-- install docker
-- copy gitlab.rb
-- partially protected
-- default project limit = 0
-- shared runner for checkstyle
-
-- protected runner for
-  
-  + setting protected tags
-  + running jplag
-
-- script for creating repos and groups
-- SSH deploy key

docs/Makefile

@@ -4,9 +4,8 @@
 # You can set these variables from the command line.
 SPHINXOPTS    =
 SPHINXBUILD   = sphinx-build
-SPHINXPROJ    = abgabesystem
-SOURCEDIR     = .
-BUILDDIR      = _build
+SOURCEDIR     = source
+BUILDDIR      = build
 
 # Put it first so that "make" without argument is like "make help".
 help:

docs/make.bat

@@ -7,9 +7,8 @@ REM Command file for Sphinx documentation
 if "%SPHINXBUILD%" == "" (
 	set SPHINXBUILD=sphinx-build
 )
-set SOURCEDIR=.
-set BUILDDIR=_build
-set SPHINXPROJ=abgabesystem
+set SOURCEDIR=source
+set BUILDDIR=build
 
 if "%1" == "" goto help
 

docs/source/conf.py

@@ -12,9 +12,9 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-import os
-import sys
-sys.path.insert(0, os.path.abspath('../src'))
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
 
 
 # -- Project information -----------------------------------------------------
@@ -26,7 +26,7 @@ author = 'Tim Schubert'
 # The short X.Y version
 version = ''
 # The full version, including alpha/beta/rc tags
-release = ''
+release = '1.0'
 
 
 # -- General configuration ---------------------------------------------------
@@ -42,7 +42,10 @@ extensions = [
     'sphinx.ext.autodoc',
     'sphinx.ext.doctest',
     'sphinx.ext.intersphinx',
+    'sphinx.ext.todo',
     'sphinx.ext.coverage',
+    'sphinx.ext.viewcode',
+    'sphinx.ext.githubpages',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -67,10 +70,10 @@ language = None
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = []
 
 # The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
+pygments_style = None
 
 
 # -- Options for HTML output -------------------------------------------------
@@ -159,9 +162,32 @@ texinfo_documents = [
 ]
 
 
+# -- Options for Epub output -------------------------------------------------
+
+# Bibliographic Dublin Core info.
+epub_title = project
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#
+# epub_identifier = ''
+
+# A unique identification for the text.
+#
+# epub_uid = ''
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+
 # -- Extension configuration -------------------------------------------------
 
 # -- Options for intersphinx extension ---------------------------------------
 
 # Example configuration for intersphinx: refer to the Python standard library.
 intersphinx_mapping = {'https://docs.python.org/': None}
+
+# -- Options for todo extension ----------------------------------------------
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True

docs/source/index.rst

@@ -1,5 +1,5 @@
 .. abgabesystem documentation master file, created by
-   sphinx-quickstart on Fri Jun  1 13:35:35 2018.
+   sphinx-quickstart on Fri Sep 28 14:59:39 2018.
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
@@ -18,18 +18,3 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
-
-.. automodule:: abgabesystem
-   :members:
-
-.. automodule:: abgabesystem.projects
-   :members:
-
-.. automodule:: abgabesystem.students
-   :members:
-
-.. automodule:: abgabesystem.commands
-   :members:
-
-.. autoclass:: abgabesystem.students.Student
-   :members:

python-gitlab.cfg

@@ -1,7 +0,0 @@
-[global]
-default = ips
-ssl_verify = true
-
-[ips]
-url = https://ips1.ibr.cs.tu-bs.de
-api_version = 4

src/abgabesystem/commands.py

@@ -1,38 +1,58 @@
 import os
 import subprocess
-import subprocess
+import logging as log
 
-from .students import Student, create_user, get_students
-from .projects import create_tag, setup_course
+from .students import Student, create_user, enroll_student, get_student_group
+from .projects import create_tag, setup_projects
 from gitlab.exceptions import GitlabCreateError, GitlabGetError
 
 
-def create_users(gl, args):
+def enroll_students(gl, args):
     """Creates Gitlab users from exported students list
+
+    Args:
+        gl: API
+        args: command line arguments
     """
+
+    student_group = get_student_group(gl, args.course)
+
     with open(args.students, encoding='iso8859') as students_csv:
         for student in Student.from_csv(students_csv):
             try:
-                create_user(gl, student, args.ldap_base, args.ldap_provider)
+                user = create_user(gl, student, args.ldap_base, args.ldap_provider)
+                # TODO this is ugly, should be group of course, but python-gitlab does not cache the query
+                enroll_student(gl, user, student_group)
             except GitlabCreateError:
                 log.warn('Failed to create user: %s' % student.user)
 
 
 def projects(gl, args):
     """Creates the projects for all course participants
+
+    Args:
+        gl: API
+        args: command line arguments
     """
-    groups = gl.groups.list(search=args.course)
-    if len(groups) == 0 and groups[0].name == args.course:
-        log.warn('This group does not exist')
+    course = None
+    for g in gl.groups.list(search=args.course):
+        if g.name == args.course:
+            course = g
+    if course is None:
+        log.warn('The course does not exist')
     else:
-        group = groups[0]
-        with open(args.deploy_key, 'r') as key, open(args.students, encoding='iso8859') as students_csv:
+        with open(args.deploy_key, 'r') as key:
             key = key.read()
-            setup_course(gl, group, students_csv, key)
+            setup_projects(gl, course, key)
 
 
 def deadline(gl, args):
-    """Checks deadlines for course and triggers deadline if it is reached"""
+    """Checks deadlines for course and triggers deadline if it is reached
+
+    Args:
+        gl: API
+        args: command line arguments
+    """
 
     deadline_name = args.tag_name
     try:
@@ -56,6 +76,10 @@ def deadline(gl, args):
 
 def plagiates(gl, args):
     """Runs the plagiarism checker (JPlag) for the solutions with a certain tag
+
+    Args:
+        gl: API
+        args: command line arguments
     """
 
     solutions_dir = 'input'
@@ -83,9 +107,13 @@ def plagiates(gl, args):
 
 def course(gl, args):
     """Creates the group for the course
+
+    Args:
+        gl: API
+        args: command line arguments
     """
     try:
-        group = gl.groups.create({
+        gl.groups.create({
             'name': args.course,
             'path': args.course,
             'visibility': 'internal',

src/abgabesystem/course.py

@@ -0,0 +1,55 @@
+import logging as log
+
+
+class InvalidCourse(Exception):
+    """Raised if the selected course is invalid.
+    """
+
+    pass
+
+
+def create_subgroup(gl, name, parent_group):
+    """Creates a group with `parent_group` as its parent.
+
+    Args:
+        gl: gitlab API object
+        name: name of the group to be created
+        parent_group: parent group of the created group
+    """
+
+    log.info("Creating subgroup %s in group %s" % (name, parent_group.name))
+    return gl.groups.create({
+        "name": name,
+        "path": name,
+        "parent_id": parent_group.id
+    })
+
+
+def create_students_group(gl, parent_group):
+    return create_subgroup(gl, "students", parent_group)
+
+
+def create_solutions_group(gl, parent_group):
+    return create_subgroup(gl, "solutions", parent_group)
+
+
+def create_course(gl, course_name):
+    """Creates a complete course as required by the `abgabesystem` including
+    the students and solutions groups.
+
+    Args:
+        gl: gitlab API object
+        course_name: name of the course, may contain any characters from
+                     [0-9,a-z,A-Z,_, ]
+    """
+
+    group = gl.groups.create({
+        "name": course_name,
+        "path": course_name.lower().replace(" ", "_"),
+        "visibility": "internal",
+    })
+    log.info("Created group %s" % course_name)
+    create_students_group(gl, group)
+    create_solutions_group(gl, group)
+
+    return group

src/abgabesystem/projects.py

@@ -1,10 +1,21 @@
+import logging as log
+
+from gitlab import DEVELOPER_ACCESS
 from gitlab.exceptions import GitlabError, GitlabCreateError
+from .students import enrolled_students
+from .course import InvalidCourse, create_solutions_group
+
 
 def create_tag(project, tag, ref):
     """Creates protected tag on ref
 
     The tag is used by the abgabesystem to mark the state of a solution at the
-    deadline
+    deadline.
+
+    Args:
+        project: GIT repository to create the tag in
+        tag: name of the tag to be created
+        ref: name of the red (branch / commit) to create the new tag on
     """
 
     print('Project %s. Creating tag %s' % (project.path, tag))
@@ -15,9 +26,17 @@ def create_tag(project, tag, ref):
     })
 
 
-
 def fork_reference(gl, reference, namespace, deploy_key):
     """Create fork of solutions for student.
+
+    Returns the created project.
+
+    Args:
+        gl: gitlab API object
+        reference: project to fork from
+        namespace: namespace to place the created project into
+        deploy_key: will be used by the abgabesystem to access the created
+                    project
     """
 
     fork = reference.forks.create({
@@ -40,6 +59,14 @@ def fork_reference(gl, reference, namespace, deploy_key):
 def create_project(gl, group, user, reference, deploy_key):
     """Creates a namespace (subgroup) and forks the project with
     the reference solutions into that namespace
+
+    Args:
+        gl: Gitlab API object
+        group: project will be created in the namespace of this group
+        user: user to add to the project as a developer
+        reference: project to fork the new project from
+        deploy_key: deploy key used by the `abgabesystem` to access the new
+                    project
     """
 
     subgroup = None
@@ -50,17 +77,18 @@ def create_project(gl, group, user, reference, deploy_key):
             'path': user.username,
             'parent_id': group.id
         })
-    except GitlabError as e:
-        subgroups = group.subgroups.list(search=user.username)
-        if len(subgroups) > 0 and subgroup[0].name == user.username:
-            subgroup = subgroups[0]
-            subgroup = gl.groups.get(subgroup.id, lazy=True)
-        else:
+    except GitlabCreateError as e:
+        for g in group.subgroups.list(search=user.username):
+            if g.name == user.username:
+                subgroup = gl.groups.get(g.id, lazy=True)
+
+        if subgroup is None:
             raise(e)
+
     try:
         subgroup.members.create({
             'user_id': user.id,
-            'access_level': gitlab.DEVELOPER_ACCESS,
+            'access_level': DEVELOPER_ACCESS,
         })
     except GitlabError:
         log.warning('Failed to add student %s to its own group' % user.username)
@@ -71,31 +99,18 @@ def create_project(gl, group, user, reference, deploy_key):
         log.warning(e.error_message)
 
 
-def setup_course(gl, group, students_csv, deploy_key):
-    """Sets up the internal structure for the group for use with the course
+def create_reference_solution(gl, namespace):
+    """Creates a new project for the reference solutions.
+
+    Args:
+        gl: gitlab API object
+        namespace: namespace to create the project in (that of the solutions for the course)
     """
-    solution = None
-    reference_project = None
 
-    try:
-        solution = gl.groups.create({
-            'name': 'solutions',
-            'path': 'solutions',
-            'parent_id': group.id,
-            'visibility': 'internal',
-        })
-    except GitlabCreateError as e:
-        log.info('Failed to create solutions group. %s' % e.error_message)
-        solutions = group.subgroups.list(search='solutions')
-        if len(solutions) > 0 and solutions[0].name == 'solutions':
-            solution = gl.groups.get(solutions[0].id, lazy=True)
-        else:
-            raise(GitlabCreateError(error_message='Failed to setup solutions subgroup'))
 
-    try:
     reference_project = gl.projects.create({
         'name': 'solutions',
-            'namespace_id': solution.id,
+        'namespace_id': namespace,
         'visibility': 'internal',
     })
     reference_project.commits.create({
@@ -109,17 +124,36 @@ def setup_course(gl, group, students_csv, deploy_key):
             },
         ]
     })
-    except GitlabCreateError as e:
-        log.info('Failed to setup group structure. %s' % e.error_message)
-        projects = solution.projects.list(search='solutions')
-        if len(projects) > 0 and projects[0].name == 'solutions':
-            reference_project = gl.projects.get(projects[0].id)
-        else:
-            raise(GitlabCreateError(error_message='Failed to setup reference solutions'))
 
-    if solution is None or reference_project is None:
-        raise(GitlabCreateError(error_message='Failed to setup course'))
+    return reference_project
 
-    for user in get_students(gl, students_csv):
-        create_project(gl, solution, user, reference_project, deploy_key)
 
+def setup_projects(gl, course, deploy_key):
+    """Sets up the internal structure for the group for use with the course.
+
+    Args:
+        gl: gitlab API object
+        course: course to set up projects for
+        deploy_key: will be used to access the solutions from the abgabesystem
+    """
+
+    solutions = None
+    solutions_groups = course.subgroups.list(search='solutions')
+    for group in solutions_groups:
+        if group.name == 'solutions':
+            solutions = gl.groups.get(group.id)
+
+    if solutions is None:
+        solutions = create_solutions_group(gl, course)
+
+    reference_project = None
+    reference_projects = solutions.projects.list(search='solutions')
+    for project in reference_projects:
+        if project.name == 'solutions':
+            reference_project = gl.projects.get(project.id)
+
+    if reference_project is None:
+        reference_project = create_reference_solution(gl, solutions.id)
+
+    for user in enrolled_students(gl, course):
+        create_project(gl, solutions, user, reference_project, deploy_key)

src/abgabesystem/students.py

@@ -1,4 +1,22 @@
 import csv
+import secrets
+
+from gitlab import GUEST_ACCESS
+
+
+class MissingStudentsGroup(Exception):
+    """Raised if a the group for the students has not already been created
+    inside the course.
+    """
+
+    pass
+
+
+class MissingCourseGroup(Exception):
+    """Raised if the group for the course is missing.
+    """
+
+    pass
 
 
 class Student():
@@ -7,6 +25,12 @@ class Student():
     Students are read from the CSV file that was exported from Stud.IP.
     For each user, a dummy LDAP user is created in Gitlab.
     Upon the first login Gitlab fetches the complete user using LDAP.
+
+    Args:
+        user: user name
+        mail: mail address of the user
+        name: full name of the user
+        group: tutorial group of the user
     """
 
     def __init__(self, user, mail, name, group):
@@ -16,7 +40,11 @@ class Student():
         self.group = group
 
     def from_csv(csvfile):
-        """Creates an iterable containing the users"""
+        """Creates an iterable containing the users
+
+        Args:
+            csvfile: CSV file from Stud.IP (latin-1)
+        """
         reader = csv.DictReader(csvfile, delimiter=';', quotechar='"')
 
         for line in reader:
@@ -24,8 +52,13 @@ class Student():
                           + ' ' + line['Nachname'], line['Gruppe'])
 
 
-def get_students(gl, students_csv):
-    """Returns already existing GitLab users for students from provided CSV file that have an account.
+def get_students_csv(gl, students_csv):
+    """Returns already existing GitLab users for students from provided CSV
+    file that have an account.
+
+    Args:
+        gl: Gitlab API object
+        students_csv: CSV file from Stud.IP
     """
 
     for student in Student.from_csv(students_csv):
@@ -34,9 +67,37 @@ def get_students(gl, students_csv):
             yield users[0]
 
 
+def enrolled_students(gl, course):
+    """Returns the students enrolled in the course
+
+    Args:
+        gl: Gitlab API object
+        course: course the students are enrolled in
+    """
+
+    students = None
+    for group in course.subgroups.list(search='students'):
+        if group.name == 'students':
+            students = group
+
+    if students is None:
+        raise MissingStudentsGroup()
+
+    # get all members excluding inherited members
+    students = gl.groups.get(students.id)
+    for member in students.members.list():
+        yield gl.users.get(member.id)
+
+
 def create_user(gl, student, ldap_base, ldap_provider):
     """Creates a GitLab user account student.
     Requires admin privileges.
+
+    Args:
+        gl: Gitlab API object
+        student: student to create user for
+        ldap_base: the search base string for the LDAP query
+        ldap_provider: LDAP provider configured for Gitlab (usually `main`)
     """
 
     user = gl.users.create({
@@ -52,3 +113,45 @@ def create_user(gl, student, ldap_base, ldap_provider):
 
     return user
 
+
+def get_student_group(gl, course_name):
+    """Gets the `students` subgroup for the course
+
+    Args:
+        gl: Gitlab API objects
+        course_name: name of the course
+    """
+
+    course = None
+    for g in gl.groups.list(search=course_name):
+        if g.name == course_name:
+            course = g
+
+    if course is None:
+        raise MissingCourseGroup()
+
+    students_group = None
+
+    for g in course.subgroups.list(search='students'):
+        if g.name == 'students':
+            students_group = gl.groups.get(g.id)
+
+    if students_group is None:
+        students_group = create_students_group(gl, course)
+
+    return students_group
+
+
+def enroll_student(gl, user, subgroup):
+    """Adds a student to the course
+
+    Args:
+        gl: Gitlab API object
+        user: user to add to the course
+        subgroup: student will become member of this group
+    """
+
+    subgroup.members.create({
+        'user_id': user.id,
+        'access_level': GUEST_ACCESS,
+    })

src/bin/abgabesystem

@@ -4,7 +4,7 @@ import gitlab
 import argparse
 import logging as log
 
-from abgabesystem.commands import create_users, projects, deadline, plagiates, course
+from abgabesystem.commands import enroll_students, projects, deadline, plagiates, course
 
 if __name__ == '__main__':
 
@@ -17,9 +17,10 @@ if __name__ == '__main__':
 
     user_parser = subparsers.add_parser(
         'users',
-        help='Creates users from LDAP')
-    user_parser.set_defaults(func=create_users)
+        help='Creates users and enrolls them in the course')
+    user_parser.set_defaults(func=enroll_students)
     user_parser.add_argument('-s', '--students', dest='students')
+    user_parser.add_argument('-c', '--course', dest='course')
     user_parser.add_argument('-b', '--ldap-base', dest='ldap_base')
     user_parser.add_argument('-p', '--ldap-provider', dest='ldap_provider')
 
@@ -35,7 +36,6 @@ if __name__ == '__main__':
     projects_parser.set_defaults(func=projects)
     projects_parser.add_argument('-c', '--course', dest='course')
     projects_parser.add_argument('-d', '--deploy-key', dest='deploy_key')
-    projects_parser.add_argument('-s', '--students', dest='students')
 
     deadline_parser = subparsers.add_parser(
         'deadline',

tests/.#test_students.py

@@ -1 +0,0 @@
-tim@metis.680:1534753335

tools/build-config.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+
+cat > python-gitlab.cfg <<EOF
+[global]
+default = default
+ssl_verify = true
+
+[default]
+url = $(echo ${CI_PROJECT_URL} | cut -d '/' -f -3)
+api_version = 4
+private_token = ${PRIVATE_API_TOKEN}
+EOF

tools/groups.py

@@ -0,0 +1,24 @@
+#!/bin/python
+
+from sys import argv
+from subprocess import run
+from os import chdir
+from os.path import isdir
+
+course_url = argv[1]
+
+with open(argv[2], 'r', encoding="latin-1") as csv:
+    for line in csv:
+        tokens = line.split(';')
+        student = tokens[5].replace('"', "").split(" ")[0]
+        group = tokens[0].replace('"', "").split(" ")[0]
+        if group != argv[3]:
+            continue
+        url = course_url + "/solutions/" + student + "/solutions"
+        path = "solutions/" + group + "/" + student
+        if isdir(path):
+            chdir(path)
+            run(["git", "pull"])
+            chdir("../../..")
+        else:
+            run(["git", "clone", url, path])