Create clearer signals when PDF fails to autogenerate

[yanlisa]

Is your feature request related to a problem? Please describe.
Our homeworks are coding + hidden written assignments, but when student PDFs aren't successfully generated (e.g. broken LaTeX/images), they receive no written Gradescope assignment score. Student workflow:

1. Students generate a zip using grader.export(). Depending on the error, the notebook generates a redbox Python warning (or not).
2. Students upload the zip to the public Gradescope autograded assignment (e.g., "Homework 1"). Because their PDF was never generated, the autograder does not submit a PDF on their behalf to the hidden Gradescope written assignment (e.g., "Homework 1 Written").
3. When grades are released, students only receive their grade for the autograded assignment ("Homework 1"). They receive no grade nor email notification for the manually graded assignment ("Homework 1 Written"). The "Homework 1 Written" assignment doesn't even show up on the student's Gradescope's assignment list.

This lack of transparency leads to many headaches for both students (who are completely unaware that there was a hidden assignment) and graders (who cannot assign a "0" grade to a student with no submission). Students also cannot submit regrade requests through Gradescope because they don't have an assignment submission.

Describe the solution you'd like
A way to signal to students that PDF failed to generate, either before or after grading. Two ideas:

• A metadata option to include a unit test such as, "Was there a successfully autogenerated PDF?" so that students can see a failed public test in their autograder.
• When the zip has no PDF, submit a blank PDF to the hidden written submission, so that teaching staff can grade as 0 then request that students submit regrades.

Describe alternatives you've considered

• A red warning block augmented to the "Your submission has been exported. Click here to download" message that a PDF was not generated, and that questions may not be graded properly.
• An automated email to the student that is sent immediately when the autograder fails to find a PDF (i.e., before grades are published). Right now students receive a Gradescope "your submission was successfully uploaded!" email upon zip upload, which is misleading.
• Manually implement a script that uploads blank PDFs for all missing submissions.

Additional context
Our teaching staff would like to avoid manual solutions where possible as our class is 1000+ students. There can be up to 10% of submissions with no autogenerated PDF, so this is a big issue for us. Thank you!

[champoll]

I have run into the same issue and would be strongly interested in a fix!

[chrispyles]

I've just merged #546 which adds a failed 0-point test when the PDF generation fails (screenshot in the PR). You can enable this behavior by setting

generate:
    warn_missing_pdf: true

in your assignment config. This will be included in v4.1.0.

• When the zip has no PDF, submit a blank PDF to the hidden written submission, so that teaching staff can grade as 0 then request that students submit regrades.

This is a more complex ask. If this behavior is still desired, please open another issue and I'll get to it in a later version, as I'm going to close this issue as a tracking issue for #546.

[champoll]

It would be really useful to have the blank PDF available. I've opened this as #551.