Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BD-27] FEAT: Add new API endpoint for uploading transcripts #27844

Merged

Conversation

kaizoku
Copy link
Contributor

@kaizoku kaizoku commented Jun 4, 2021

Description

This adds another view and URL route for an endpoint which accepts transcript uploads. As opposed to the existing endpoints, this accepts JWT tokens as credentials. The existing endpoints only work for currently logged in users. This fills a need to programmatically upload transcripts from external scripts.

This doesn't alter existing behavior, but does extend and re-use code from the existing transcript_upload_handler() view.

Useful information to include:

  • Which edX user roles will this change impact? Course Authors and Developers

Supporting information

This was needed for use on openedx/cc2olx#61. Initial discussion for the change began there.

Testing instructions

  1. Create a test video with edx video ID in the devstack, note edx video ID for step Fix/vik/speed up oe test #3.
  2. Get JWT access token: curl -d 'token_type=jwt&grant_type=client_credentials&client_id=xxxxxxxxxxxxxxxxxxx&client_secret=xxxxxxxxxxxxxx' http://localhost:18000/oauth2/access_token
  3. Upload transcript: curl -H 'Authorizations: JWT xxxxxxxxxx' http://localhost:18010/upload_transcript_api/ -F file=@${HOME}/devstack/edx-platform/common/test/data/uploads/uk_transcripts.srt -F language_code=en -F edx_video_id=ae5f4ec8-7b27-447b-b094-080f74657e03

Deadline

June 7th, 2021

Other information

Include anything else that will help reviewers and consumers understand the change.

  • Does this change depend on other changes elsewhere?
    No, but will be utilized in cc2olx.
  • Any special concerns or limitations? For example: deprecations, migrations, security, or accessibility.
    The naming in the repurposing of validate_transcript_upload_data() feels a bit awkward, but in the pursuit of DRY it seems better to re-use the form validation as these are very similar endpoints.

cc: @alangsto

@openedx-webhooks
Copy link

openedx-webhooks commented Jun 4, 2021

Thanks for the pull request, @kaizoku! I've created BLENDED-876 to keep track of it in Jira. More details are on the BD-27 project page.

When this pull request is ready, tag your edX technical lead.

@openedx-webhooks openedx-webhooks added open-source-contribution PR author is not from Axim or 2U waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. labels Jun 4, 2021
@alangsto alangsto changed the title WIP: FEAT: Add new API endpoint for uploading transcripts [BD-27] WIP: FEAT: Add new API endpoint for uploading transcripts Jun 4, 2021
@openedx-webhooks openedx-webhooks added blended PR is managed through 2U's blended developmnt program and removed open-source-contribution PR author is not from Axim or 2U labels Jun 4, 2021
@kaizoku kaizoku force-pushed the josh/BB-4323-transcript-upload-api branch from 3827262 to d311154 Compare June 7, 2021 16:36
@kaizoku kaizoku changed the title [BD-27] WIP: FEAT: Add new API endpoint for uploading transcripts [BD-27] FEAT: Add new API endpoint for uploading transcripts Jun 7, 2021
@openedx-webhooks openedx-webhooks added needs triage and removed waiting on author PR author needs to resolve review requests, answer questions, fix tests, etc. labels Jun 7, 2021
@alangsto
Copy link
Contributor

alangsto commented Jun 7, 2021

@edx/masters-devs-cosmonauts tagging for more visibility. This PR is needed for improvements to the video/transcript upload tool that is part of CC2OLX. Alan had to make similar changes to allow for video upload using API credentials in https://github.com/edx/edx-platform/pull/25513.

Copy link
Contributor

@schenedx schenedx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested to add logging statements. Rest looks straightforward.

)
response = JsonResponse(status=201)
except (TranscriptsGenerationException, UnicodeDecodeError):
response = JsonResponse(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please log this error!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a log message here. Does the language look alright?

},
file_data=ContentFile(sjson_subs),
)
response = JsonResponse(status=201)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we please add logging statement for the success case? It really helps monitoring

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a message here as well, also including the edX video ID and transcript language.

@alangsto
Copy link
Contributor

alangsto commented Jun 7, 2021

jenkins run python

1 similar comment
@alangsto
Copy link
Contributor

alangsto commented Jun 7, 2021

jenkins run python

@kaizoku
Copy link
Contributor Author

kaizoku commented Jun 7, 2021

Thanks for the review @schenedx . Should I add some log messages to the existing input validation function as well?

If not, and the log messages I've just added look good I can squash the extra commits here.

@edx-status-bot
Copy link

Your PR has finished running tests. There were no failures.

@alangsto
Copy link
Contributor

alangsto commented Jun 8, 2021

@kaizoku Thanks for adding those additional log messages. I don't think you need to add log messages to the existing input validation function, as errors that are caught in that function will ultimately be returned by the new endpoint you are adding.

@alangsto alangsto merged commit bc082ea into openedx:master Jun 8, 2021
@edx-pipeline-bot
Copy link
Contributor

EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production.

@edx-pipeline-bot
Copy link
Contributor

EdX Release Notice: This PR has been deployed to the production environment.

@0x29a 0x29a deleted the josh/BB-4323-transcript-upload-api branch June 22, 2021 11:27
blarghmatey pushed a commit to mitodl/edx-platform that referenced this pull request Aug 2, 2021
…#27844)

* FEAT: Add new API endpoint for uploading transcripts

* Add log messages to transcript upload function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blended PR is managed through 2U's blended developmnt program merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants