Welcome to Quickscope’s documentation!¶
Overview¶
Quickscope is used to build Gradescope autograders with Chalkbox, the University of Queensland ITEE school’s automatic marking system for programming assignments.
Quickscope is hosted at quickscope.uqcloud.net for public consumption.
Key Features¶
Support for Python and Java based programming assessment.
Extensible engine system for introducing new languages and environments.
Dependency and environment management with Poetry.
A friendly web interface:
Build and Run¶
Quickscope requires the Poetry dependency and environment management system as well as Python 3.8. If you don’t have Poetry installed run:
pip install poetry
If you don’t have Python 3.8 we’d suggest using the pyenv management system to install it:
pyenv install 3.8.2
Once you have those, you can install Quickscope:
git clone https://github.com/UQTools/quickscope
cd quickscope
poetry shell
poetry install
poetry run build
When the generation of the front end is complete you can run the Quickscope server:
poetry run flask run
You should now be able to access the service through your browser at 0.0.0.0:5000
.
Documentation¶
Documentation for the project is available at quickscope.readthedocs.org. This includes guides on how to use the tool and adapt it to your needs.
Copyright & License¶
Quickscope is copyright Nicholas Lambourne, Max Miller & Ella de Lore.
The Quickscope logo is copyright Nicholas Lambourne.
This tool has been licensed for general use under a permissive MIT license available here.
Guides¶
Guides for students, tutors and course coordinators using Quickscope.
Creating a JavaEngine Autograder¶
This guide explains the process of configuring and generating an autograder bundle in Quickscope, using the Java Engine for ChalkBox.
Generic Setup¶
Visit http://quickscope.uqcloud.net and log in with your UQ credentials.
Choose a course code and assignment identifier for the autograder. These will be displayed in the instructor-facing output for debugging purposes each time the autograder runs.
Choose the JavaEngine under the ‘Engine’ dropdown box. A set of engine-specific options will appear below.

Java Engine Configuration¶
Here, you can select the individual stages to be run as part of the autograder. All stages are optional and can be run separately or together with one or more other stages. Selecting the checkbox next to the name of a stage will enable it, and any stage-specific options will appear below.
Each stage has a weighting, which can be changed by entering a number in the associated text field. This represents the total number of marks, out of 100, allocated to this stage. A submission that scores perfectly in a given stage will receive the number of marks entered here. If a weighting is 0, the stage will still be run, however no marks will be allocated to it.
Immediately below the options for each stage are file drop zones for dependency libraries and the correct solution directory, explained below.

Any dependency JARs that are required by the assignment should be added here, for example, JUnit and Hamcrest. If the Checkstyle stage is enabled, the Checkstyle JAR should be added here as well.

The directory containing the assignment solution should be dragged here. This
directory should contain the two directories src
and test
, along with any
extra text files provided by course staff for the purposes of testing
I/O-related classes.
For example, if the directory to be added to Quickscope is called correct
,
the structure should be:
correct
├── saves
│ └── example.txt
├── src
│ └── tms
│ ├── display
│ │ └── SimpleDisplay.java
│ ├── intersection
│ │ └── Intersection.java
│ ├── route
│ │ ├── Route.java
| ... more files
└── test
└── tms
├── intersection
│ └── IntersectionTest.java
├── route
│ ├── RouteTest.java
... more files
The correct solution directory should contain all the solution code for the
assignment inside src/
, as well as all the tests to be run against the
student submission (if the functionality stage is enabled) in test/
.
To provide immediate feedback to students on submission, a subset of the
uploaded JUnit tests in the test/
directory can be marked as visible at the
time of student submission, rather than the default of only after the grades
have been published/finalised for the assignment.
Immediately visible tests will appear in the results section when a student
makes a submission, along with the mark received for that test and the test
output if it failed.
To mark an individual JUnit test as visible, use the @Deprecated
annotation.
For example:
@Test
@Deprecated
public void toStringTest() {
assertEquals("A", intersection1.toString());
}
Weightings can be used to allocate more marks to a particular unit test where it is deemed necessary.
The weighting of each individual JUnit test can be modified by adding the
timeout
parameter to the @Test
annotation. Weightings are integers between
one and nine inclusive, representing a weighting of one and nine times the
regular test weighting respectively.
When including the timeout
parameter of a test to specify a weighting, it is
recommended to add a sufficiently large value such that the test does not
actually time out for the given weighting, such as one million milliseconds.
For example, to give a test a weighting of three times the regular weighting:
@Test(timeout = 1000000 + 3)
public void hashCodeTest() {
assertEquals(a.hashCode(), b.hashCode());
}
Note that the recommended approach to adding timeouts to tests is to use a class-wide timeout rule as shown below.
public class MyTest {
@Rule
public Timeout timeout = Timeout.seconds(1);
...
After enabling the conformance stage by clicking the checkbox under Java Engine Configuration, the violation penalty can be changed by entering a number in the input field. This number represents the number of marks deducted for each instance of non-conformance to the expected public API found in the correct solution. The total of all deductions is capped at the weighting assigned to the stage, meaning students cannot receive a negative mark.
A file drop zone is provided for specifying the expected file structure of a student submission. This should generally be similar to the correct solution directory, however, only the tests classes that are assessable should be present. This is to avoid extra test classes being marked as ‘missing’ in the conformance check.
For example, if the directory to be added to Quickscope is called
correct_structure
and the assessable test classes are IntersectionTest
and
DemoPressurePadTest
, the structure should be:
correct_structure
├── saves
│ └── example.txt
├── src
│ └── tms
│ ├── display
│ │ └── SimpleDisplay.java
│ ├── intersection
│ │ └── Intersection.java
│ ├── route
│ │ ├── Route.java
| ... more source files
└── test
└── tms
├── intersection
│ └── IntersectionTest.java
└── sensors
└── DemoPressurePadTest.java
The JUnit classes that are assessable in the assignment should be selected by clicking on the rounded boxes representing files under the “Preview:” section below the correct solution drop zone.
Once a file has been selected, it will appear in the list of Assessable Test Classes. To remove a file from the list, simply click its rounded box again.

Below the list of assessable test classes, there is a file drop zone for
uploading a directory containing a set of faulty solutions to the assignment,
used when assessing submitted JUnit tests. The correct implementation of the
assignment should also be included in this directory, under solution/
.
As a suggestion, each subdirectory should be named according to the test class
it is assessing. For example, pp
for DemoPressurePadTest
.
These subdirectories should act as the src/
directory for each solution,
with packages as immediate subdirectories.
For example, if the directory to be added to Quickscope is called
faulty
, the structure should be:
faulty
├── pp_getCongestion_calc
│ ├── README.txt
│ └── tms
│ ├── display
│ │ └── SimpleDisplay.java
│ ├── intersection
│ │ └── Intersection.java
│ ├── route
│ │ ├── Route.java
│ ... more source files
├── ... more faulty solutions
└── solution
└── tms
├── display
│ └── SimpleDisplay.java
├── intersection
│ └── Intersection.java
├── route
│ ├── Route.java
... more source files
After enabling the Checkstyle stage by clicking the checkbox under Java Engine
Configuration, the violation penalty can be changed by entering a number in the
input field. This number represents the number of marks deducted for each
style violation detected by the Checkstyle tool when run on the src/
directory of the submitted assignment.
A text input field is provided for specifying paths that should be excluded from being checked for style violations by the Checkstyle tool. Excluded paths can be directories (using a trailing slash) or individual files. If a directory is specified, the entire directory will be ignored when checking for style violations. This functionality can be useful when the assignment contains code provided by course staff that is not compliant with the style guide, for example, a package containing GUI-related code.
To add an excluded path, enter the path in the input field, ensuring the path
is relative to the root directory of the submission, /autograder/submission/
.
Then, click the “+” button inside the input field.
The path will display under “Excluded Paths”.
A file drop zone is provided to specify the configuration file to be used
by the Checkstyle tool.
This is an .xml
file containing a list of all violation types to detect,
and any associated options.

Generating and Uploading Autograder¶
Finally, once all stages have been configured, the autograder is ready to be
generated.
Simply click the “Generate Autograder Bundle” button at the bottom of the page,
and a .zip
file containing the entire autograder suite will be downloaded
automatically.
This file can then be uploaded to Gradescope.
Creating a PythonEngine Autograder¶
This guide explains the process of configuring and generating an autograder bundle in Quickscope, using the Python Engine for ChalkBox.
Generic Setup¶
Visit http://quickscope.uqcloud.net and log in with your UQ credentials.
Choose a course code and assignment identifier for the autograder. These will be displayed in the instructor-facing output for debugging purposes each time the autograder runs.
Choose the PythonEngine under the ‘Engine’ dropdown box. A set of engine-specific options will appear below.

Python Engine Configuration¶
Here you will be required to provide the needed files for the engine.
You will first have to provide the expected name of the student submitted file (e.g a1
)
and the name of test script (e.g test_a1.py
).

You will need to provide a folder of included files to Quickscope containing:
The test script and dependencies (e.g. for CSSE1001,
test_a1.py
andtestrunner.py
)Support files for testing (e.g. config files)
Support files for student code (any files that have been provided to students as support code)
This is an example of an included folder for CSSE1001, using the 2019 assignment 1:
included
│
├── test_data
│ │
│ ├── main_fixed_win.in
│ │
│ ├── main_fixed_win.out
│ │
│ ... more files
│
├── a1_test.py
│
├── testrunner.py
│
├── a1_support.py
│
├── WORDS_ARBITRARY.txt
│
└── WORDS_FIXED.txt
You will need to provide a text file that contains the names of test classes that will be available to students before the due date. The results from the tests named in the file will be visible to students each time they upload a submission to Gradescope.
Generating and Uploading Autograder¶
Finally, once all stages have been configured, the autograder is ready to be
generated.
Simply click the “Generate Autograder Bundle” button at the bottom of the page,
and a .zip
file containing the entire autograder suite will be downloaded
automatically.
This file can then be uploaded to Gradescope.
Setting up an Assignment in Gradescope¶
This guide explains the process of creating a programming assignment in Gradescope, using an autograder generated by Quickscope.
Creating the Assignment¶
Firstly, follow the steps outlined in Gradescope’s demonstration video below to create a programming assignment using the Gradescope interface.

Assignment Settings¶
The following options should be set in the “Assignment Settings” panel when creating the assignment:
Autograder points should be set to the total of the weights of all the automatically marked components of the assignment. This number should be the maximum possible number of marks achievable in the autograder.
For JavaEngine, this will be the sum of all the weights assigned to the JavaEngine stages.
For PythonEngine, this will be the maximum achievable functionality score, excluding the manually marked style component.
“Enable Manual Grading” should be enabled if there is to be a manually marked style component
“Due Date” should be set to the official due date of the assignment. Submissions made after this date but before the Late Due Date will be marked as “Late” when marking, along with the number of days it is late.
“Allow late submissions” should be enabled.
“Late Due Date” should be set to the last expected submission date of the longest extension that is allowed for the assignment. Students will not be allowed to submit after this date. Course staff can still submit on behalf of students, though students cannot see their autograder results on submission.
Outline¶
In the “Outline” section, an additional question should be created for the manual style component, including the weighting assigned to manual style.

Autograder¶
Finally, the autograder .zip
generated by Quickscope should be uploaded on
this page by selecting “Zip file upload”, “Select Autograder (.zip)” then
“Update Autograder”.
After uploading the autograder, you can test that it works correctly by clicking “Test Autograder” and uploading a student submission, for example, the correct solution for the assignment written by course staff.

Other Settings¶
By clicking the “Settings” link at the bottom of the left sidebar after entering the assignment page, other various settings can be enabled that are not present in the “Assignment Settings” panel shown when creating the assignment. Key changes that can be made here include:
Specifying which submission methods are allowed. Submission via direct file upload, GitHub and Bitbucket can be enabled or disabled.
Specifying ignored files. Files that match these conditions will be filtered out from a student’s submission before being processed with ChalkBox. This may be useful if ChalkBox is having problems with certain types of files.
Changing the autograder container’s system specifications. More CPU and RAM may improve performance.
Changing the autograder timeout. The default timeout is 10 minutes before an autograder instance is killed, however this can be increased up to 40 minutes.

Style Marking in Gradescope¶
This guide covers the process of style marking student assignments in Gradescope.
Marking¶
The Gradescope marking interface uses a number of shortcut to assist with marking. Please watch the following videos to learn about the interface and these shortcuts.


Commenting¶
Comments can be added to student code by clicking on a section of student code. A comment box will then appear where you can add your comment and then save.

The student will be able to view comments linked to the code that you clicked on.
Shortcuts¶
1 - 9
- Select Rubric GroupQWERTY
- Select Rubric ItemLeft Arrow
- Previous StudentRight Arrow
- Next Studentz
- Next Ungradeda
- Show all submissions.
- Next Question,
- Previous Question
Understanding Gradescope Feedback CSSE2002¶
When you have submitted your assignment to Gradescope, you will receive feedback on you work. If you submit before the due date you will receive some “pre-checks” on you assigment. Full feedback will be available at the time of grade release.
Before Due Date¶
Before the due date of the assignment, submitting your assignment to Gradescope will allow you to receive “pre-checks” on your submitted code.

These checks are:
Compilation - checks if your program compiles, if not, give the compilation error output explaining why your code failed to compile.
Conformance - checks if your program conforms to the provided specification.
Functionality - a small number of selected functionality tests may be provided for you to see the functionality of your program. These tests are a subset of the full suite of tests used to mark your assignment’s functionality.
JUnit Test Compilation - checks if your provided JUnit tests compile with our solution. If your tests do not compile with our solution then you will receive 0 marks for the JUnit section of the assignment
Style - automatically checks your program for style violations. These violations will be deducted from your total style marks.
Assignment Feedback¶
When the assignment grades are published you will be able to view you final grades and feedback. The screen will be similar to the feedback explained above, except there will be three main differences.

Full Functionality - You will be able to see the results from all the functionality tests for the assignment and whether you passed or failed each test.
JUnit Results - You will be able to see the results from your tests. Your tests are run against the solution to the assignment and a number of faulty solutions. Your mark is determined by the number of faulty solutions that pass fewer of your tests than are passed by the actual solution. The name in parentheses tells you which of your test classes that faulty solution was accessing, along with a brief identifier for the faulty solution.
Style Feedback - You can view your full style feedback by navigating to the “Code” tab on the results page.
This will show you your manual style grade. To see detailed feedback, click on the question name in green (in this case, “Manual Style”) which will expand to show the grading rubric and deductions. Files with comments will have a speech box containing the number of comments in that file. To view the comments, click on the file name to expand it.
Submitting to Gradescope¶
Using Gradescope you are able to submit files either by uploading them or through a GitHub repository.
To find the submission portal for Gradescope you will need to navigate to the dashboard for the subject where you will see a list of assignments.

By clicking on the assignment you wish to submit, the submission portal will appear.

File Upload¶
If you wish to upload a file, select the upload radio button and either drag the file from your computer or click on the drag & drop square to open a file explorer.
If you are submitting a file structure with directories you will need to zip them to preserve the structure.
Specifically for CSSE2002 you will need to zip your top level folders
(src, test etc) together. In the below example you wouldn’t compress a1
,
you need to select all of the sub folders of a1 (in this case src and test)
and then compress them together.
a1
├── src
│ └── tms
│ ├── display
│ │ └── ... files
│ ├── intersection
│ │ └── ... files
│ ├── route
│ │ └── ... files
│ ├── sensors
│ │ └── ... files
│ └── util
│ │ └── ... files
└── test
└── tms
├── intersection
│ └── IntersectionTest.java
└── sensors
└── DemoPressurePadTest.java
GitHub¶
If you wish to submit an assignment that is in a GitHub repo you will need to select the GitHub radio button.

You will then be prompted to connect your GitHub account to Gradescope.

You will then be prompted to select the repository and branch of the assignment you wish to submit.
Submission¶
Once you have selected your options and clicked the Upload
button,
you will be taken to the results page. The results will not appear instantly as there may be some
processing time. If your course has pre submission feedback you will be able to
see it there.
Quickscope API¶
Details of the Quickscope API class, functions and methods.
quickscope¶
quickscope package¶
-
quickscope.server.bundle.
copy_testrunner
(bundle_directory)¶ PythonEngine specific. Copies the testrunner.py testing utility package from ../templates into the included directory in the bundle.
- Parameters
bundle_directory (
Path
) – the temporary directory where the bundle is being created- Return type
None
-
quickscope.server.bundle.
get_chalkbox
(version, bundle_directory)¶ Procures the specified version of ChalkBox from GitHub releases.
- Parameters
version (
str
) – the version of ChalkBox to get (e.g. v0.2.0)bundle_directory (
Path
) – the path to the temporary directory in which the bundle is constructed
- Return type
Path
- Returns
the path to the ChalkBox JAR in the temporary bundle directory.
-
quickscope.server.bundle.
get_dependencies
(dependency_directory)¶ Creates a list of dependencies based on the contents of the dependency directory.
- Parameters
dependency_directory (
Path
) – where the dependencies uploaded by the user are stored- Return type
List
[str
]- Returns
a list of the dependencies’ paths (including the dependency directory)
-
quickscope.server.bundle.
produce_bundle
(config)¶ Performs bundle construction from the various elements based on the Engine and configuration specified.
- Parameters
config (
Dict
[str
,Any
]) – the configuration dictionary as prepared by .templates.populate_config- Return type
str
- Returns
the path to the zipped bundle as a string
-
quickscope.server.bundle.
produce_config_file
(form, bundle_directory)¶ Takes the form from the React front-end and uses it, in combination with the engine default settings, to populate the configuration and write it to the config.yaml file in the bundle directory.
- Parameters
form (
Dict
[str
,Any
]) – the immutable form dictionary from the request object populated by the React front-endbundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_included_directory
(source, bundle_directory)¶ PythonEngine specific. Copies the entire ‘included’ directory with its uploaded components into the root of the bundle.
- Parameters
source (
Path
) – the original location of the included directory as uploaded by the userbundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_lib_directory
(lib_directory, bundle_directory)¶ JavaEngine specific. Gathers the lib file containing JAR dependencies and copies it to the temporary bundle directory.
- Parameters
lib_directory (
Path
) – the lib directory with JAR dependencies uploaded by the userbundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_resources_directory
(resources_directory, bundle_directory)¶ JavaEngine specific. Gathers the static resources directory and files uploaded by the user and copies them to the temporary bundle directory.
- Parameters
resources_directory (
Path
) – the static resources directory populated by the user’s uploadsbundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_run_script
(run_call, bundle_directory=None)¶ Creates the run.sh script in the root of the bundle that is used by Gradescope to start the autograding process.
- Parameters
run_call (
str
) – typically the call to start ChalkBox with whatever arguments are requiredbundle_directory (
Optional
[Path
]) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_setup_script
(setup_calls, bundle_directory)¶ Creates the setup.sh script - required by Gradescope to prepare the Ubuntu environment - and places it in the bundle.
- Parameters
setup_calls (
str
) – the calls made to e.g. install packages, set the PATH etc. These come from .templates.pybundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
produce_solution_directory
(solution_directory, bundle_directory)¶ Copies the solution directory uploaded by the user to the temporary bundle directory.
- Parameters
solution_directory (
Path
) – the directory containing the correct and faulty solutions uploaded by the userbundle_directory (
Path
) – the temporary directory where the bundle is being created
- Return type
None
-
quickscope.server.bundle.
reformat_test_classes
(config, session_directory)¶ JavaEngine specific. Transforms the assessable test classes listed in the config from Java import style (e.g. chalkbox.import.style) to path style (e.g. chalkbox/import/style.java).
- Parameters
config (
Dict
[str
,Any
]) – the config dictionary containing the test classes to updatesession_directory (
Path
) – the directory associated with the user’s session where the user’s uploads are stored
- Return type
None
-
quickscope.server.routes.
generate
()¶ Generates the bundle based on the uploaded files and the configuration settings passed through in the form.
- Return type
Response
- Returns
a response that downloads the generated bundle to the client machine
-
quickscope.server.routes.
home
()¶ Serve the React front end bundle.
- Return type
str
- Returns
the rendered template of the React index.html page
-
quickscope.server.routes.
upload_locations
(component)¶ Uploads a file to the correct location in the appropriate state directory (based on the session id and the component type).
- Parameters
component (
str
) – the component of the bundle that is being uploaded, this will determine the subdirectory in the state directory based on the engine- Return type
Response
- Returns
a response object indicating success or failure
Utility class for running Quickscope
-
quickscope.server.templates.
populate_config
(config, form, session_directory, locations)¶ Takes the user-provided configuration settings from the form and uses them to update the default values for the specified engine. User settings will override defaults.
- Parameters
config (
Dict
[str
,Any
]) – the basic settings common to all configurations: course code, assignment ID, engine, and session directoryform (
Any
) – the form (immutable dictionary) attached to the Flask request containing the user-specified settingssession_directory (
Path
) – the path to the directory associated with the user session matching the session ID created by the React front-endlocations (
Dict
[str
,str
]) – the mapping from required components to their respective location in the bundle (e.g. PYTHON_LOCATIONS or JAVA_LOCATIONS, above)
- Return type
Dict
[str
,Any
]- Returns
the compiled configuration dictionary
-
quickscope.server.utils.
collapse_path_overlap
(clean_file, component, locations)¶ Removes the risk of creating duplicated folders in the state directory by checking if the clean_file has any directory overlap with the component location e.g. /solutions/correct/ and correct/a1.py would ensure that a1.py was put in /solutions/correct/a1.py and not /solutions/correct/correct/a1.py.
- Parameters
clean_file (
str
) – the uploaded file cleaned of any leading forward slashescomponent (
str
) – the particular component being uploadedlocations (
Dict
[str
,str
]) – the set of upload locations specific to the engine associated with the session
- Return type
str
- Returns
the collapsed path as a string
-
quickscope.server.utils.
deep_update
(original, updates)¶ Update a nested dictionary with new values, leaving unupdated values in place. Modifies original in place.
- Parameters
original (
Dict
) – the dictionary whose values will be updatedupdates (
Mapping
) – the dictionary with the values to you want to insert into original
- Return type
Dict
-
quickscope.server.utils.
make_session
(session_id)¶ Creates a session directory with the given session_id if it does not already exist.
- Parameters
session_id (
str
) – the session ID generated by the React front-end whenever the page is loaded or reloaded.- Return type
Path
- Returns
the path object corresponding to the session_id, which now must exist.
-
quickscope.server.utils.
reconstruct
(session_id, component, files, locations)¶ Finds the appropriate directory for uploaded file(s) and saves them there.
- Parameters
session_id (
str
) – the session ID provided by the React front endcomponent (
str
) – the type of component being uploaded (e.g. linter config file)files – the files from the Flask request
locations (
Dict
[str
,str
]) – the locations associated with the components that make up the engine selection
- Return type
None
-
quickscope.build.
build
()¶ Installs front-end dependencies and builds the front-end using yarn, which must already be installed.
- Return type
None