Instructional Support

The Instructional Support application provides examination management and TA assignment services to course and administrative staff. This document provides a high-level overview of the capabilities of the application. More detailed documentation is also available:

  • Help — Documentation for application users.
  • System Documentation — Detailed information for system administrators and developers (very incomplete so far).

This system works for classes of all sizes, although most of its features exist because of large classes. If all classes were small there would be much less need for a system of this type.

Statistics

Statistics of this nature have the potential to be misleading, but they may provide some sense of the overall magnitude of the application.

  • Python code (interface and scheduled processes): 6400 lines
  • SQL code (schema definition and stored procedures): 8000 lines
  • Java code (PDF handling): 4800 lines

This application depends on several other applications: Mail Sending, Printing, Identity, Quest, WatIAM.

Examination Management

Examination management includes informing candidates when they need to write examinations as well as where they should go to write. This includes individually assigned seats. The system can also print examination papers personalized for each candidate and sorted conveniently for setup.

Basic Configuration

In order to maximize automation and minimize how much work has to be done by the staff for each course, the system automatically maintains instructor and class registration data based on Quest. Additionally, the system includes a single database of rooms and seats.

Every day the instructor and class registration information is updated based on a Quest data feed. Because class administration may be organized in ways not recorded in Quest, the system uses its own concept of “Admin Unit” instead of Quest courses. Quest courses may be held together or, even if not held together, managed together. Conversely, Quest courses may be split up by instructor or by other criteria, with some sections being run separately from others.

Course staff are responsible for setting up midterm examinations within the system. This includes entering the start time and rooms, booked via existing processes external to the system. Final examinations are automatically imported from the final examination schedule.

Sittings

Typically, at least a few candidates will write an examination at different times or places than most of the candidates. The system is designed to support a wide variety of scenarios via the concept of “sittings”, including the following:

  • AAS (AccessAbility Services) — Candidates with special needs write with AccessAbility Services, usually in Needles Hall or nearby locations. AAS provides information about which students are affected directly to the system, so all course staff need to do is send the appropriate examination papers to AAS in time.
  • Relief — Candidates with conflicting or consecutive final examinations may change the time of one of them. This is managed by the Registrar’s Office, which provides a spreadsheet of affected candidates well in advance of the beginning of the final examination period.
  • Conflicts — Candidates with a legitimate conflict with a midterm time may write at a different time. This is typically organized by course staff, who can create additional sittings at the required times.
  • Multiple Locations — CEL (formerly Distance Education) examinations, such as CS 100, are held at multiple locations, with assigned seating in use for candidates writing on-campus in Waterloo, which are a majority of the total.
  • In-class Midterms — The system does not specifically support this use case, but it can still be handled by setting up sittings at the appropriate class times and using the interface to allocate each section’s candidates to the appropriate sitting.

Seat Assignments

As mentioned, the system gives course staff the option of assigning candidates to individual seats. This makes it easier for candidates to find their seat: instead of having to search around, possibly through several rooms or even buildings, for an available space, all they need to do is go to their assigned seat. Experience proves that this works much better than directing people to buildings or rooms, for example by last name, which tends to have poor compliance.

Assigned seating also has examination integrity benefits. Candidates sit next to whom the system decides randomly, rather than near friends who may be tempted to look to them for answers.

Given the set of rooms booked for the examination, the system attempts to determine the exact seats to use automatically based on the set of candidates selected for the examination. It will attempt to avoid using as many as possible of the smallest rooms. After that it will spread out the candidates among the rooms, filling smaller rooms while leaving extra space in the larger rooms.

Course staff can configure which seats will be used via several features:

  • Per-examination configuration: by default, tablet-arm seats will not be used, and candidates will be seated in every other seat in most rooms (in locations such as the PAC where there is sufficient separation, every seat will be used regardless). Either of these rules can be overridden.
  • Per-room cramming: a number of additional candidates may be “crammed” into a room, up to the specified cram limit. The cram limit defaults to 0. The system will attempt to spread out the crammed-in candidates as much as possible within the rooms in which cramming is allowed.
  • Seat reservation: any seats may be individually marked as “Reserved”, which means they will not be affected by any automatic processes. In particular, they will not be used by seat assignment.

The system communicates seat assignments to candidates through a single URL used for all examinations. Candidates may log in and see a list of their upcoming examinations including seating locations.

Sequence Numbers & Ordering

The order used to assign seats can be configured by course staff. By default seats are assigned randomly but candidates can be sorted by instructor and section and/or by name depending on what is convenient.

Each examination paper is assigned a sequence number, which is a short code that is used for sorting the examination papers. The order of assignment of the sequence numbers can be configured. The sequence numbers are printed on the examination papers, allowing them to be easily slotted into the correct folder after the examination and sorted into order. This feature is most useful in large classes where keeping hundreds of examination papers organized can be difficult.

Printing

In order to handle printing of examination papers, the system can accept PDF examination master uploads from an appropriate member of the course staff. They can also set the permitted aids. This information along with personalized information for each examination candidate is imposed on the uploaded PDF and turned into a print job which is automatically sent to New Media Services.

Submission to New Media Services of print requisitions and files for printing is handled by the Print application.

Administrative Documents

The system allows course staff to download PDFs of documents to assist with examination administration. These include:

  • Posting list — This is a list of seating locations organized by student ID so that candidates who have forgotten their exact seat location can still find it.
  • Sequence lookup — This is for use after the examination to convert student ID numbers into sequence numbers to make finding a particular examination easy.
  • Folders — Ledger size inserts for hanging file folders to hold the examination papers. Eash insert is labelled and includes a list of exactly which examination papers it should contain.
  • Labels — Personalized labels with the same candidate information as the cover of the examination papers. These may be printed on standard label stock if needed and are for labelling anything else, such as answer booklets, that need to be individually labelled.
  • Proctor Package — A room map and attendance list for each room. The room map is to assist with setup. The attendance list is for proctors to use when checking candidate identification and includes a space for each candidate to sign.

Staff can also download a CSV file listing the candidates for the examination. This is meant to assist with mark entry.

TA Assignment

In addition to examination management, the Instructional Support application also supports the management of TA assignments.

For initial setup, authorized individuals within the Graduate office can set up the course offerings for which TAs (and IAs, etc.) are required, and how many units of each are needed. Similarly, they can configure which students should be given TA units, and how many. Finally, they can set the dates upon which the information becomes visible to course staff and graduate students.

Once the relevant courses and students are set up within the system, authorized staff can view and update the TA assignments. The system has views which attempt to show anomalies or areas in which more work is needed. In particular, courses with too few or too many TA units assigned compared to the plan, and students with too few or too many TA units assigned compared to the plan, are easy to find.

Authorized staff can configure two dates for visibility of the information: Information becomes visible to course staff and graduate students when the “tentative” date is reached, but no promise is made that the information will not change; there is also a “finalized” date after which changes may still be made but affected course staff and graduate students should be specifically informed.

Course staff may see their TA information at the bottom of the main page for each course offering. Graduate students will see their TA assignments at the top of the front page for the instructional support application.

Integration

This application integrates with other parts of the University in several ways. Some of these are mentioned above. This is meant to be a convenient list of them collected together:

  • AccessAbility Services — The system automatically emails AAS to inform them which classes’ information we need; AAS then logs in an uploads the relevant information.
  • Registrar Relief — Once a term, the Registrar’s Office sends Relief information by email which is then uploaded into the system.
  • Centre for Extended Learning — CEL uploads information concerning where their students are writing directly into the system. The system then assigns seats and produces cover page print jobs.
  • New Media Services — Print jobs are sent (using the “Print” application) to New Media Services. These include the CEL cover pages, the PAC posting lists, and of course the actual examination papers.
  • Data Import — Class sections and student class registrations are automatically imported from Quest (using the ”Quest” application). Additionally, the final examination schedule is automatically imported from Open Data.
  • Registrar’s Office Printing — Final examinations that would otherwise be printed by the Registrar’s Office may be printed through this system. For each examination submission deadline, the relevant examination print jobs are created and sent to New Media Services. They are then included in the regular deliveries of examination papers to distribution points along with other examinations.